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ieee RODUCTILON 


A. BACKGROUND 

The Navy is currently testing a new source selection tool 
known aS Red Yellow Green (RYG). The test is being conducted 
at five Navy field activities (Naval Air Engineering Center 
Lakehurst, Naval Avionics Center Indianapolis, Naval Ships 
Parts Control Center Mechanicsburg (Code 021, Level 1/SS), 
Naval Supply Center Charleston/Naval Shipyard Charleston, and 
Naval Supply Center Pensacola/Naval Aviation Depot Pensacola). 
The purpose of the RYG program is to assist Government 
contracting personnel in obtaining the best purchase value by 
adding the costs of poor contractor performance into the 
Source selection equation [Ref. 1}. The RYG program was 
designed especially for use by field contracting activities 
[Ref. 2}. 

In addition to RYG, several other contractor quality 
programs are currently being employed or developed within the 
Department of Defense (DOD) that consider contractor quality 
and past performance in the source selection process. Three 
notable quality programs are the Blue Ribbon Contractor 
program, the Contractor Performance Assessment Reporting 
System (CPARS), and Total Quality Management (TQM). This 
thesis will describe the four programs listed above. It will 


then evaluate the RYG program by comparing and contrasting it 


to the other three programs, discussing its strengths and 


weaknesses, and describing the status of the program test. 


B. ‘OBO pCi vis 

DOD and its agencies have continuously sought ways to 
improve the quality of Government contractor performance while 
complying with the statutes, regulations, and policies that 
govern the Federal acquisition process. Recent problems with 
the quality of material delivered to DOD, many of them 
highlighted in the press, have increased DOD's emphasis on 
contractor quality management. At the same time, steadily 
increasing Congressional oversight in the form of more and 
increasingly detailed statutes and guidance have made 
development of a viable program more difficult. Some of the 
programs currently employed or under review by DOD commands 
include the Navy's RYG program, Blue Ribbon Contractor 
programs within the Navy and Air Force, the Air Force's CPARS, 
and the DOD-wide TOM initiative. 

One of these programs, the Navy's RYG, is the subject of 
this thesis. This program is intended to eventually be 
extended to all DOD agencies. It uses the data contained in 
the Navy's newly established unified, centralized Contractor 
Evaluation System (CES). These data are compiled from many 
sources including Quality Deficiency Reports (QDRs) and the 
Naval Material Quality Assessment Office (NMQAO). (Ref. “aa 


Prior to CES, these data were found in many different forms 


and each came from a different source. Now, they are 
available in a single location and can be accessed by field 
procurement activities from a computer terminal using an on- 
line call-up menu. CES is used to classify contractors (by 
Federal Supply Class (FSC)) as high risk quality performers 
(Red), moderate risk (Yellow), or low risk (Green). 
Contractors who do not meet established criteria for RYG 
classification are listed in the "Insufficient Data" 
category. (Ref. l:p. 1] The RYG classification is used by 
contracting personnel to either apply Technical Evaluation 
Adjustments (TEAS) to contractor proposals [Ref. l:p. 1] or, 
if the Fixed Price-Greatest Value method is employed, to rate 
the offerors in terms of expected quality of performance [Ref. 
l:encl (3), p. 3]. The TEAS represent the anticipated cost to 
the Government to correct or take appropriate action due to 
Beer contractor performance [Ref. l:encl (1), pp. 1-2]. 
Proposals including TEAs are used to help determine a source 
selection that will result in the best overall price to the 
mevernment (Ref. l:encl (1), p. 6]. 

The RYG program is undergoing a one-year test coordinated 
by the Office of Assistant Secretary of the Navy (Shipbuilding 
and Logistics) Reliability, Maintainability, and Quality 
Assurance (ASN(S&L)RM&QA) which began 1 August 1989 using a 


limited number of commodities. 


C. THE RESEARCH QUESTIONS 
The primary research question is: How is the Navy's Red 
Yellow Green program structured and how is it intended to 
improve the quality of material procured by the Navy? The 
Subsidiary research questions are: 
1. How are Red, Yellow, and Green contractors defined? 


2. What are the other current contractor “quale, 
initiatives within DOD? 


3. How does RYG compare and contrast to these other 
programs? 


4. What are the strengths and weaknesses associated with 
RYG? 


5. How is the RYG program being tested in the market? 


D. SCOPE AND LIMTPATIONS 

The purpose of this thesis is to evaluate the potential 
effectiveness of the Navy's RYG program. It will analyze the 
program in detail and describe other DOD contractor quality 
initiatives in brief. It will compare and contrast the RYG 
program to the other DOD initiatives and will attempt to 
identify RYG strengths and weaknesses. It will evaluate the 
success of RYG tests as far as they have progressed. Finally, 
it will consider the viability of the RYG program in the 
future, both within the Navy and DOD. 

The research effort was limited by the short period of 
time over which the test of RYG could be evaluated. Although 
some test activities were able to begin implementation of the 


RYG program on 1 August 1989, others were delayed. This 


thesis evaluated the program very early in the test period, 


and, therefore, relied on limited data currently available. 


Pee THODOLOGY 

Two methods were used to collect data and answer the 
research questions. Data were collected from secondary 
sources to obtain information about RYG and other DOD contrac- 
tor guality initiatives. Data were collected from primary 
sources, specifically through telephone and personal inter- 
views, to add to the above areas and to assess the current 
status of the test and the potential for future expansion of 
the program. The interviews involved personnel at 
ASN(S&L)RM&QA, NMQAO, the Naval Supply Systems Command, and 


mimi Tye activities involved in the test. 


E. ORGANIZATION OF THE STUDY 
The remainder of the thesis is organized into the follow- 
ie chapters: 


1. Chapter II, "Red Yellow Green Program," will describe 
the RYG program concept and how it 1s being coordinated. 


2. Chapter III, “Other Department of Defense Quality 
Initiatives," will briefly describe other current DOD 
contractor quality programs in place or under review. 


3. Chapter IV, "A comparison of Red Yellow Green and Other 
Contractor Quality Initiatives," will compare and 
contrast the RYG program to the quality programs 
described in Chapter III. 


4. Chapter V, "Strengths and Weaknesses of Red Yellow 
Green," will describe the advantages and potential 
problems and limitations associated with RYG. It will 
also discuss the status of the RYG test, describing how 


the program test iS progressing and attempting to 


project 


Chapter 
discuss 
quality 
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program. 


how the program will fare in the future. 


VI, "Conclusions and Recommendations," will 
whether RYG will be effective in improving the 
of material procured by the Navy and will 
on possible changes for weak areas of the 


eee yt LD LOW GREEN PROGRAM 


A. BACKGROUND 
The Navy's Contractor Evaluation System (CES) and Red 
Yellow Green (RYG) program were established to both assist 
Navy activities in procuring quality goods and services and to 
comply with Department of Defense (DOD) and Secretary of the 
Navy (SECNAV) policy. SPeCciElcCally spe. Directive s4155.4 
states: 
DOD Components shall assure that contracts are not awarded 
to contractors with a previous history of providing products 
or services of an unsatisfactory quality. Contractor 
quality history data shall be maintained and used for this 
purpose. Ske rom op. 9 1 =] 
Further, SECNAV Instruction 4855.7 states: 
It is Secretary of the Navy policy that: 

a. Contractor quality history data be collected and 
maintained in a single Navy CES. 

b. Contractor quality history data be used to assure that 
contracts are not awarded to contractors with a previous 
history of providing products of an unsatisfactory quality, 
and to determine actions necessary before and after contract 
awards to assure product quality. ([Ref. 4] 


The CES and RYG program satisfy the requirements of these 


policies. 


Eee CONTRACTOR EVALUATION SYSTEM 

iVemehowls currently composed of data from the Product 
Deficiency Reporting and Evaluation Program (PDREP) data base. 
The Navy has plans to expand the CES to include other general 


information about contractors such as financial data in 


addition to information concerning contractor Gua, ae ae 
expanded CES will be called the Buyer Information System. It 
Will be available to Government contracting personnel to help 
them better evaluate offerors. At present, however, the 
CES/PDREP is a quality reporting system managed by the Naval 
Sea Systems Command (NAVSEA) Detachment, Naval Material 
Quality Assessment Office (NMQAO), under the direction of the 
Office of the Assistant Secretary of the Navy (Shipbuilding 
and Logistics) Reliability, Maintainability, and Quality 
Assurance (ASN(S&L) RM&QA). [Ref. 3:p. v] 
PDREP was established to satisfy the requirements of 
SECNAV Instruction 4855.1A for "a product deficiency reporting 
and data feedback system, maintenance of contractor/supplier 
quality history and effective use of the data to influence the 
pre-contract award process and formulate the basis for 
necessary post award quality assurance actions." [Ret. "oy 
The purpose of PDREP is to 
...-implement a product deficiency reporting and evaluation 
program that will feed data back to those activities respon- 
Sible for design, development, purchasing, production, 
supply, maintenance, contract administration, and other 
acquisition functions and initiate action to appraise, 
correct, and prevent product deficiencies, decrease material 
ownership costs, evaluate contractor/supplier performance 
and use performance data for making procurement decisions. 
(Od 07> seen ca 

PDREP's purpose, then, is compatible with the goals of the RYG 


program. Therefore, it was a logical data base to be chosen 


for the Navy's new CES which was intended to _ be used to 


categorize contractors as Red, Yellow, or Green within a given 


Federal Supply Class (FSC). 


The CES is composed of contractor quality information 


collected from the sources which are described below. 


In 


Quality Deficiency Reports (QDRs). QDRs are prepared by 
Navy field activities to document product quality 
deficiencies, design deficiencies, or inadequate 
procurement documents resulting in defective new and 
newly reworked material being delivered to the Navy 
[Ref. 6]. All QDRs are submitted to Naval Air Systems 
Command (NAVAIR), the Navy focal point for QDRs. Once 
each week QDRs determined to be contractor liable and 
with defects verified are transmitted by NAVAIR to NMQAO 
for inclusion in PDREP. [Ref. 7] 


Material Inspection Record (MIR). MIRS are prepared 
either by Navy representatives performing technical 
inspections at a contractor's plant or by Navy field 
activities performing technical inspections upon receipt 
of material. MIRs are submitted to the Navy Systems 
Command having cognizance over the field activity. 
[Ref. 3:p. 1-5} The Systems Commands (NAVAIR, NAVSUP, 
Naval Facilities Engineering Command, and Space and 
Naval Warfare Command) then transmit the MIRs to NMQAO. 
The MIRs generated by NAVSEA activities are submitted 
directly to NMQAO. ([Ref. 7] 


Reports of Discrepancy (RODs). RODS are prepared by 
Navy field activities to document receipt of incorrect 
material, shortages and overages, and discrepancies in 
preservation, packing, and marking [Ref. 6:p. 1-2}. 
RODS are submitted to Naval Supply Systems Command 


(NAVSUP). Procedures are currently being developed to 
transmit RODs from NAVSUP to NMQAO for inclusion in 
PDREP. However, at present they are not being 


transmitted and, therefore, are not yet part of PDREP. 
pret. 7] 


Defense Logistics Agency (DLA) Contractor Improvement 
Program (CIP) Alert List. Contractors are placed on the 
DLA alert list if DLA has placed them in the CIP, if 
they have received a negative pre-award survey, or if 
Defense Contract Administration Services (DCAS) has 
recommended they be given a pre-award survey for a 
particular reason [Ref. 8]. DLA sends a hard copy of 
the list to ASN(S&L)RM&QA. ASN then sends a copy to 
NMQAO. [Ref. 7] 


DLA Method cC, D, and E Corrective AcCtiemee ise ina 
Contractors are placed on the corrective action listing 
if DLA has documented deficiencies in their quality 
programs. Specifically, method C indicates that the 
contractor hasS a serious quality problem or has not 
corrected a deficiency documented using method B (a 
major deficiency). The Government sends a letter to the 
firm's top management requesting corrective action. 
Method D indicates that less severe methods of 
corrective action (1i.e., A, B, and C) have failed. The 
acquisition quality assurance program is discontinued, 
and the contractor is advised that the Government will 
not accept his goods or services until deficiencies have 
been corrected. Method E is used to advise a prime 
contractor that a subcontractor has quality deficiencies 
that would justify method C or D corrective action ina 
prime contractor and to request that the prime take 
corrective action with his subcontractor. [Ref. 9] DLA 
sends a hard copy of the listing to ASN(S&L)RM&QOA. 
ASN then sends a copy to NMQAO. [Ref. 7] 


Navy Vendor Data Analysis Report (VDAR). The VDAR 
identifies contractors who, because of past poor 
performance, should be considered carefully before being 
awarded a contract and should be monitored after 
contract award. Evaluation of performance is based on 
data from pre-award surveys, product oriented surveys, 
quality system reviews, QDRs, open DLA method C, D, or 
E corrective action, and conviction of or investigation 
for malpractice or fraud. [Ref. 8:p. E-5] The VDAR is 
compiled by NMQAO based on past performance and input 
from Navy Systems Commands and their field activities 
[Refs 10] 5: 


Pre-Award Surveys. Pre-award surveys are conducted by 
contract administration offices when a _- procuring 
contracting officer needs additional information to 
determine contractor responsibility. The survey 
evaluates the CON EAGCEO rs management, financial 
capability, and technical capability to determine 
whether he will be able to perform the proposed 
COnERaAcE. (Ret... 115) Only those pre-award surveys 
requested by Navy activities are included in PDREP. The 
Navy activity that requested the survey submits a copy 
to the cognizant Systems Command. The Systems Commands 
then transmit copies to NMQAO. [Ref. 10] NAVSEA 
activities submit copies of surveys directly to NMQAO 
PRe female 


Product-Oriented Surveys. Product-oriented surveys are 
technical product inspections conducted in a 


10 


eo". 


contractor's plant when a buying activity desires to 
perform a special test on an item. They are performed 
by DCAS when requested by the buying activity. If DCAS 
does not have the necessary resources, the buying 
activity may perform the survey. When a Navy activity 
requests a product-oriented survey, it submits a copy to 
the appropriate Systems Command. The Systems Commands 
then transmit the surveys to NMQAO. [Ref. 10] NAVSEA 
activities submit copies of surveys directly to NMQAO 
mefto7 |. 


Quality System Reviews. Quality system reviews are 
performed by DCAS. They involve an evaluation of the 
contractor's quality procedures and verification that 
the contractor's quality practices conform to those 
procedures. [Ref. 9:p. 23] The reviews also evaluate 
the Government's in-plant quality assurance program. 
Navy activities receive copies of quality system reviews 
if they participate in the review with DCAS or if they 
request a copy. (Ref. 10] Copies received by Navy 
activities are submitted to the appropriate Systems 
Command. The Systems Commands then transmit the reviews 
to NMQAO. NAVSEA activities submit copies directly to 
NMQAO. [Ref. 7] 


Certificates of Competency (COCs). If a small business 
is determined to be non-responsible by a Government 
buying activity, the small business can request that the 
Small Business Administration (SBA) determine whether 
the business is responsible. If the SBA concludes that 
the small business is responsible, it will prepare a COC 
to document that determination. The buying activity 
must then treat the small business as a responsible 
offeror. [Ref. 3:p. 1-8] The SBA sends hard copies of 
COCs to NMQAO for inclusion in PDREP [Ref. 7]. 


The CES data base excludes: 


Material evaluations for base application and local use. 


Contractors developing major weapon systems. 


Medical procurements, material, suppliers, or 
evaluations. 
Subsistence procurements, material, suppliers, Or 
evaluations. 


Unsatisfactory material condition caused by improper 
handling after receipt, deterioration during local 
storage, or inadequate maintenance or operation. 


Ja 


6. Transportation discrepancies caused by the carrier. 


7. Ammunition and explosives accidents. 

8. Nuclear weapon procurements, material, suppliers, or 
evaluations. 

9. Naval Nuclear Power Plant primary system procurements, 


material, suppliers, or evaluations. 
10. Strategic Systems Project Office procurements, 
. suppliers, or material evaluations. [Ref. 3:p. 1-3} 
C. RED YELLOW GREEN PROCEDURES 
A one year test of the RYG program commenced 1 August 
1989. Tt is being conducted at the following five Navy 
activities: Naval Air Engineering Center Lakehurst, Naval 
Avionics Center Indianapolis, Naval Ships Parts Control Center 
(SPCC) Mechanicsburg (Code 021, Level 1/SS), Naval Supply 
Center Charleston/Naval Shipyard Charleston, and Naval Supply 
Center Pensacola/Naval Aviation Depot Pensacola. [Ref. 2] 
The RYG program is a source selection tool that uses 
information documented in the CES to evaluate contractor past 
performance. Based on information in the CES, contractors are 
classified by FSC as high risk quality performers (Red), 
moderate risk (Yellow), or low risk (Green). New contractors 
or those for which the CES does not have enough data are 
Classified as "Insufficient Data." Because the classification 
is by FSC and contractor, one contractor may have several 
color classifications if he supplies the Government with more 


than one commodity. (Ref. l:sencl. (1), p. 3] 


Ae 


The general descriptions of the color classifications as 
outlined in the program procedures are: 

1. Red. The contractor's performance history indicates he 
has supplied goods or services in a particular FSC of 
poor enough quality to warrant requirement of special 
quality actions and higher level review before contract 
award. He is designated as a high quality risk. 

2. Yellow. The contractor's performance history indicates 
he has supplied goods or services ina particular FSC of 
poor enough quality to warrant requirement of special 
quality actions. He is designated as a moderate quality 
risk. 

3. Green. The contractor's performance history indicates 
that he has supplied high quality goods or services and, 
therefore, is designated as a low quality risk. His 
proposal is evaluated without anticipating special 
Guality actions. [Ref. Lrencl. (1), pp. 3-4] 

Contractors are classified as Red, Yellow, or Green based on 
the specific criteria listed in Appendix A [Ref. l:encl. (1), 
attachment (2)}]. The RYG data base will be updated monthly to 
reflect a particular contractor's current classification [Ref. 
ieeomecl., (1), p. 5). 

If no negative quality performance data are received on 
Red or Yellow contractors during the previous 12 months, they 
will be reclassified. Red contractors will automatically be 
reclassified as Yellow, and Yellow contractors will be placed 
in the Insufficient Data classification. If a contractor was 
classified as Red or Yellow because of his reject rate on 
MIRs, and if no rejects have occurred on the inspection of 
five or more lots of material during the past six months, he 


will be reclassified as Yellow or Green respectively. Ifa 


contracting officer has reason to question a commodity 
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classification, NMQAO should be contacted for confirmation of 
the current classification. (Ret. enei())) eee! 

Contractor classifications will be updated by NMQAO 
monthly based on new data received. After each update, NMQAO 
will send a computer tape to each of the five test activities 
reflecting the new contractor classifications. The most 
current data will then be used by the activities in 
performance of the test. [Ref. 12] 

The RYG test procedures stress that a contractor's color 
classification should never keep him from competing for a 
Government contract. They specifically state: 

The procedures set forth herein do not in any way obliterate 
the requirement that a written responsibility determination 
is to be made for every requirement prior to award. A 
contractor's color classification alone 1s not sufficient 1ee 
determine that a contractor 1s or 1s not responsible without 
further consideration of the standards set forth in FAR 
9.104. (Ref. “1: enc] mele pi. maw 

The detailed procedures of eHe RYG program are divided 
into three categories, simplified small purchase, major 
purchase, and fixed price-greatest value procedures. A 
description of the application and specific requirements of 
each of these categories follows. 

1. Simplified Small Purchase Procedures 

The simplified small purchase procedures will be 
tested at all five test activities. They apply to all oral or 
written quotations solicited during the one year test for 


selected FSCs that will result in purchase orders with an 


estimated value greater than $2500. The FSCs used for the 
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test were chosen by each test activity. They may be modified 
during the test by agreement between the test activity and 
NMQAO. Each activity can choose whether to employ these 
procedures for purchases under blanket purchase agreements, 
imprest fund purchases, and delivery orders against 
established contracts OF with the General Services 
Administration. However, if RYG procedures are used for these 
purchases, the activity must issue a written or confirming 
purchase order to document the purchase for the file. Waiver 
of the use of these procedures may be granted by the Chief of 
the Contracting Office for all test sites except SPCC. At 
SPcc, the Director of the Hull, Mechanical, and Electrical 
Contracting Department must grant any waivers. [Ref. l:encl. 
(1), pp. 1-2) 

When a purchase made under these procedures requires 
a synopsis in the Commerce Business Daily, the notice will 
explain that, although price will be a significant factor in 
pmemposal evaluation, other factors, including the contractor's 
quality history, will also influence the award decision. 
Likewise, the solicitation document must include a clause 
explaining the RYG program to prospective offerors and 
advising them of its use for the solicited procurement. ([Ref. 
Meencl. (1), p. 6) The clauses to be used for simplified 
small purchase procedures are provided in Appendix B [Ref. 


meemel. (1), attachment (6)]}. 


1) 


Since the basis of a contractor's color clacewriecactenm 
by FSC does not vary, the level of quality risk is the same 
regardless of the procedures used for the procurement (1.e., 
small purchase, major procurement, or greatest value). 
However, the procedures that contracting personnel follow 
after receiving proposals does vary. Under the simplified 
small purchase procedures, the following procedures are 
followed. 

First, each of fergie s Classification must be 
determined from the data base--Red, Yellow, Green, or 
Insufficient Data. If the contractor is classified Green, his 
proposal iS evaluated as received since past performance 
indicates that quality should not be a concern if he is 
selected. If the offeror is classified Yellow, the 
appropriate TEA is added to his quoted price. The adjusted 
price should better represent what the actual cost to the 
Government would be (considering the cost to correct quality 
problems) if he were selected. If the “contractor a= 
classified Red, again his quoted price is adjusted by adding 
an appropriate TEA. This TEA will be higher than the one 
assigned to a Yellow contractor since the costs the Government 
incurs while doing business with a Red contractor will 
probably be greater than that of a contractor classified as 
Yellow. If there are insufficient data to assign the 


contractor a color classification for the FSG involved ine wen 


eS 


See rtation, there 1S no basis for adding a TEA to his cost 
proposal, so none is assigned. [Ref. l:encl. (1), pp. 3-4] 

After TEAS are assigned to proposals from Red and 
Yellow contractors, adjusted prices are used in the source 
selection process. If a contract 1S awarded to other than a 
Green contractor, additional actions must be taken by the 
purchasing activity during the test period. It is anticipated 
that similar actions will be required when RYG is exported for 
general use by procuring activities. If award is made to a 
Red contractor, for example, the head of the contracting 
office must approve the award. The RYG test procedures 
stress, however, that under no circumstances is a Red 
contractor to be barred from competing for a contract. ([Ref. 
l:encl. (1), p. 4) If award is made to either a Red or Yellow 
contractor, contracting personnel should ensure that any 
necessary quality control requirements are included in the 
purchase order [{Ref. l:encl. (1), p. 6]. If award is made to 
a contractor on which the Government has insufficient quality 
data, even though no TEAS were assigned, the procuring 
activity may choose to reguire greater quality controls than 
they would if dealing with a Green contractor [Ref. l:encl. 
(1), p.- 4). 

The TEAs for the simplified small purchase procedures 
are set at standard values [Ref. l:encl. (1), p. 2]. The TEA 
for a Red contractor is $2499 and for a Yellow contractor is 


S255 (Ref. l:encl. (1), attachment (4)]. These values are 


aS 


based on the anticipated requirement for the following 
additional Government quality actions. For a Red contractor, 
additional requirements would be Government source inspection 
(GSI), receipt inspection at destination performed with a Navy 
representative present, and a quality assurance letter of 
instruction. For a Yellow contractor, the requirements would 
be GSI and a quality assurance letter of instruction. /[Ref. 
ivene., (1), attachment (3) ] The dollar values were 
calculated by estimating the costs the Government would incur 
in performing these activities. Specifically, GSI is assigned 
a cost of $500 by estimating this would take the Government 14 
hours at $34.18 per hour. Receipt inspection at destination 
1s expected to cost the Government $1194 based on eight hours 
at $43 per hour plus $200 for material handling and $650 for 
testing. Finally, the quality assurance letter of instruction 
is estimated to cost $755 based on eight hours at $34.18 per 
hour for DCAS personnel plus 16 hours at $30 per hour for 
procurement representative personnel. fRef. l:encl. (2 
attachment (3) ] 
2. Major Purchase Procedures with TEAs 

The major purchase procedures with TEAs will be used 
at only two of the five test activities, SPCC Mechanicsbitam 
(Code 021, Level I/SS) and Naval Supply Center Pensacola/Naval 
Aviation Depot Pensacola. These procedures apply to all 
negotiated competitive solicitations with an estimated value 


greater than $2500 for the FSCs covered by the test during the 
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one year test period when large purchase procedures are used. 
As with the simplified small purchase procedures, each test 
activity has chosen the FSCs to be analyzed during the test, 
and these may be modified by agreement between the test site 
and NMQAO. Waiver of the procedures may be granted by the 
emmeere of the Contracting Office or, at SPCC, by the Director 
Semeetne Hull, Mechanical, and Electrical Contracting 
Department. teeteeleencl. (2) >) pp... 2=3)] 

When a procurement covered by these test procedures 
requires a synopsis in the Commerce Business Daily, the notice 
will explain that, although price will be a significant factor 
in proposal evaluation, other factors as detailed in the 
Swietreitation, including the offeror's quality history, will 
influence the award decision [Ref. l:encl. (2), p. 7}. The 
solicitation will include essentially the following notice to 
offerors: 

This procurement is part of a test of the Navy's Contractor 
Evaluation System, "Red/Yellow/Green" Program. Award will 
be based upon the Contracting Officer's decision as to which 
offer provides the best value to the Navy--price, past 
quality performance, and other factors considered. Details 
are provided in the provisions entitled "NOTICE TO 
PROSPECTIVE OFFERORS" (Section L) and "ADDITIONAL EVALUATION 
mee lORS—--TEST OF CONTRACTOR EVALUATION Sedat (MAJOR 
PURCHASE PROCEDURES)" (Section M). [Ref. l:encl. (2), p. 7] 
The clauses to be included in Sections L and M of the 
solicitation are provided in Appendix C [Ref. l:encl. (2), 
attachment (5)]}. 


When proposals are received, contracting personnel 


query the RYG data base to determine the color classification 


ies. 


of each offeror for all FSCs covered by the solicitation. As 
with Simplified small purchase procedures, a Green 
classification indicates that the offeror is a low quality 
risk and, therefore, no TEAs are assigned to his proposal. If 
the contractor has a Yellow classification, the procurement 
package and the contractor's quality history will be used to 
determine necessary Government quality assurance actions and 
associated TEAs. The TEAS are then added to the offeror's 
proposed price. If the contractor has a Red classification, 
appropriate TEAs are determined in a manner similar to those 
for Yellow contractors. However, a greater number of quality 
assurance actions will be expected when dealing with a Red 
COnEeBaAe Lor. Therefore, the TEAs assigned will probably be 
higher. If the contractor does not have ae color 
classification because of insufficient data, he will not be 
assigned any TEAs. [Ref. l:encl. (2), p. 4] Additionally, if 
the procurement is from a sole source supplier, no TEAs will 
be assessed regardless of the applicable color classification 
[| R@f sk ene larG2) eer nc 

The TEAs for major purchase procedures are based on 
the specific quality deficiencies in the performance history 
of the contractor for a particular FSC. These deficiencies 
are used to determine which additional quality assurance 
actions the Government should take to ensure receipt of a 


quality good or service. [Ref< Di: enciw (2), oe | 


Ze 


Contracting personnel take the following steps when assigning 
TEAS under major purchase procedures. 

First, contracting officers ascertain the reason for 
the contractor's classification being Red or Yellow by pulling 
his classification code from the RYG data base (listed in 
Appendix A). Second, they decide which additional quality 
assurance actions the Government must take. A guideline is 
provided to assist them (Appendix D) [Ref. l:encl. (2), 
attachments (3)&(4)]. As can be seen in Appendix D, the 
contractor's deficiency code or codes are tied to an alpha- 
numeric code for additional quality assurance requirements. 
Now, contracting personnel perform the final step, assignment 
of TEAs. The alpha-numeric codes for quality assurance 
actions correspond to the list of quality assurance 
requirements and associated TEAS in Appendix D. The TEAs 
listed are calculated in the same way as those used in the 
Simplified small purchase procedures. Specific calculations 
for each TEA are detailed in Appendix D. The test procedures 
stress, however, that the costs on which the listed TEAs are 
based are provided as examples. Since labor rates and test 
costs may vary considerably between procuring activities, each 
test activity should recalculate the TEAs to reflect the 
procuring activity's estimated costs. 

After TEAs are assigned to appropriate proposals, all 
proposals are evaluated together (using adjusted prices for 


@enmtractor's assigned a TEA) [Ref. l:encl. (2), p- 6]. If the 
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contract is awarded to a Red or Yellow contractor, the 
contracting officer must ensure that the additional quality 
assurance actions used to calculate TEAs are called out as 
requirements in the contract [Ref. l:encl. (2), p. 7]. If the 
contract is awarded to a Red offeror, approval for the award 
must be granted by the head of the contracting office [Ref. 
l:encl. (2), p. 4]. If a contractor on which the Government 
has insufficient quality data wins the award, the procuring 
activity will require appropriate quality assurance actions to 
ensure a certain quality level even though TEAS were not 


assigned (Ref. l:encl. (2), p. 4]. 


3. Major Purchase Procedures with Fixed Price-Greatest 
Value 


The greatest value procedures will be used at only 
three of the five test activities; Naval Air and Engineering 
Center Lakehurst, Naval Avionics Center Indianapolis, and 
Naval Supply Center Charleston/Naval Shipyard Charleston [Ref. 
l:encl. (3), p. 2}. These procedures apply to all negotiated 
competitive solicitations with an estimated value greater than 
$2500 for the FSCs covered by the test during the one year 
test period where sealed bidding procedures would otherwise be 
most appropriate [Ref. l:encl. (3), p. 2}. Each test activity 
has chosen the FSCs that will be covered by the test; these 
may be modified by agreement between the activity and NMQAO 


(Ref. l:encl. (3), p- 3]. Waiver of the procedures may he 
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feemtea by the Chief of the Contracting Office [Ref. l:encl. 
(3), p-. 3]. 

Solicitations prepared under these procedures will 
include the same notice used under the major purchase 
Beocedures employing TEAS [Ref. l:encl. (3), p. 7]. The 
clauses to be included in Sections L and M of the solicitation 
are provided in Appendix E [Ref. 1:encl. (3), attachment (3)]. 
The source selection/evaluation plan will be prepared by the 
Quality staff and will specify the criteria to be used in 
evaluating Red, Yellow, and Green commodities. This plan will 
be forwarded with the procurement request ae Ene Cehcbhace Ing 
officer for review and approval. Reto tsencl., (3), p. 3) 
For those commodities covered by these procedures, the source 
selection/evaluation plan will assign price a minimum weight 
of 40 percent, with the remainder assigned to quality 
assurance considerations under RYG procedures [Ref. l:encl. 
(3), p. 2]. During the test, no factors other than these two 
shall be considered without prior approval from Naval Supply 
Systems Command (NAVSUP) Code 02 via NAVSUP Code 021 ([Ref. 
meme. (3), p. 2]. Appendix F ViRef. L:encl.” (3); p. 8] 
provides an example of the RYG evaluation portion of the 
source selection/evaluation method. 

If a contractor is determined to be Green, he will 
normally be rated as "Excellent" or "Good." A Yellow 
mrvssittcacion Will normally carry with it a rating of "Good," 


"Average," or "Marginal." Red commodities will usually be 
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rated "Marginal" or "Unacceptable." If there are insufficient 
data to evaluate the offeror, the proposal will be considered 
technically acceptable and will be evaluated solely on the 
basis of price. The contracting officer can exercise 
discretion in the use of adjectives (1i.e., excellent, good, 
etc.) for each color classification or other locally 
determined source selection criteria. [Ref. l:encl. (3), pp. 
4-5] 

When proposals are received, the contracting office 
will forward the names and Commercial and Government Entity 
(CAGE) codes of all offerors to the Quality staff. The 
Quality staff will determine the color classification and the 
associated reason codes for each offeror for each FSC. They 
will then assign ratings to each offeror based on the criteria 
outlined in the source selection plan. If commodities of 
different offerors lie within the same color classification, 
their relative differences will be explained in an 
accompanying narrative. This evaluation will be forwarded to 
the contracting office. [Ref. J 2enel. (soe 

If the contract is awarded to a Red or Yellow 
contractor, the Quality staff will determine any necessary 
additional quality assurance requirements that should be 
imposed ony sthe “contractor. The contracting officer wie 
ensure that these are included in the contract. [Ref. l:encl. 


(3), p- 3) If the contract 1S awarded to a Red contractor, 
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approval for the award must be granted by the head of the 


Semeracting office (Ref. l:encl. (3), p. 3]. 


D. SUMMARY 

This chapter described the Navy's RYG program. First, it 
explained reasons behind establishment of the program. 
Second, it described the CES data base on which the program is 
based. Finally, it detailed the test procedures that are 
being used to implement the program. The next chapter will 
briefly describe three other current DOD contractor quality 


programs. 


ae, 


Ill. OTHER DEPARTMENT OF UDEPENSE OUALITY Shira Ly ho 


Ne INTRODUCTION 

One factor in the researcher's evaluation of the Red 
Yellow Green (RYG) program iS an assessment of how this 
program compares with other contractor quality programs being 
used or considered for use by the Navy. Three programs 
evaluated by the researcher are the Air Force Contractor 
Performance Assessment Reporting System (CPARS), the Naval 
Avionics Center (NAC) Indianapolis Blue Ribbon Contractor 
program, and the Department of Defense (DOD) Total Quality 
Management (TQM) initiative. These three programs were chosen 
for the assessment because each has been considered for use 
by the Navy as an alternative to RYG. Therefore, a discussion 
of the merits and shortcomings of these programs and their 
relationship to RYG is considered important. The evaluation 
described above will be the subject of Chapter IV of this 
thesis. In preparation for the assessment in Chapter IV, this 
chapter provides background information on the procedures 


associated with each of the three programs. 


Be CONTRACTOR PERFORMANCE ASSESSMENT REPORTING SYSTEM 
The first program evaluated is CPARS used by the Air 
Force. As stated in Air Force Systems Command (AFSC) 


Regulation 800-54, "The sole purpose of CPARS is to provide 
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for a command-wide performance data base used in AFSC source 
selections." (Ref. 13] CPARS currently includes 8:4 
contractors and their divisions and subsidiaries. These 
contractors were chosen because the Air Force conducts most of 
its business with them. Therefore, it was determined that the 
benefits of maintaining performance data on them would be 
worth the effort involved in the CPARS process. [Ref. 14] A 
Contractor Performance Assessment Report (CPAR) iS prepared 
on contracts with the 84 contractors included in the program 
valued at more than $5 million. Preparation of the reports 
are further limited to contracts for concept demonstration and 
validation, full-scale development, and full-rate production 
and deployment. The program excludes contracts for laboratory 
work, services, and operations and maintenance. [Ref. 13:p. 
1) Following are the areas of contractor performance 
evaluated in the CPAR: 
a. Product system performance. 
a. Engineering design/support; 
b. Software development. 


2s Schedule. 


3. Cost performance. 
as Product assurance. 
5. Test and evaluation. 


6. Integrated logistics support program. 
7. Management responsiveness. 


8. Subcontract management. 
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9. Variances. 
a. Cost; 
b. Schedule; 
10. Other areas unique to the contrac evaluated. [Ref. 15] 

The CPAR is prepared by the program director or program 
manager. An initial CPAR for a new contract is prepared 
between six months and one year after contract award. 
Following this, CPARS are submitted annually during the 
contract performance period. Additional CPARs are completed 
when the program manager changes or if the program manager 
determines that a contractor's performance has' changed 
sufficiently to warrant modification of the most recent CPAR. 
In addition, the contractor can request that a CPAR be 
completed before the annual report is prepared if he believes 
that his performance has changed enough to alter the 
assessment. The final CPAR is prepared within six months 
after the final major end item is delivered or when a contract 
is terminated. [Ref. 13:p. 2] 

Preparation of the CPAR involves four steps. First, the 
program manager or engineer responsible for the contract 
prepares a preliminary assessment with associated 
documentation. Second, the program manager forwards the 
original of the preliminary CPAR to the contractor and retains 
a Copy. The contractor has 30 days to respond to the CPAR. 
Upon receiving a response from the contractor or 30 days after 
transmitting the CPAR to the contractor, whichever occurs 


first, the program manager takes the third step and signs the 
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CPAR. He may first revise his evaluation if the contractor's 
response indicates that a change is appropriate. Finally, the 
program manager sends the signed CPAR with its attachments to 
the Product Division of the AFSC having cognizance over his 
program for input into the command-wide data base. (Ref. 
sep, 2 | 

As part of his assessment, the program manager translates 
his evaluation of the contractor in each of the ten 
performance evaluation areas into a color classification. The 
following is a description of the criteria he must use in 
aSSigning colors: 


Blue (Exceptional). Indicates performance clearly 
exceeds contractual requirements. The area of evaluation 
contains few minor problems for which corrective actions 
appear highly effective. For cost performance, blue 
indicates a positive cost variance. 


Green (Satisfactory). Indicates performance clearly 
meets contractual requirements. The area of evaluation 
contains some minor problems for which corrective actions 
appear satisfactory. For cost performance, green indicates 
no cost variance or a negative cost variance greater than 
zero but less than or equal to five percent. 


Yellow (Marginal). Indicates performance meets 
contractual requirements. The area of evaluation contains 
a serious problem for which corrective actions have not yet 
been identified, appear only marginally effective, or have 
not been fully implemented. For cost performance, yellow 
indicates a negative cost variance greater than five percent 
PitemetLessS than or equal to 15 percent. 


Red (Unsatisfactory). Indicates the contractor is in 
danger of not being able to satisfy contractual requirements 
and recovery is not likely ina timely manner. The area of 
evaluation contains serious problems for which corrective 
actions appear ineffective. For cost performance, red 
indicates a negative cost variance greater than 15 
percent. PReibe 133 p.. 7] 
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Since each area is evaluated by color, a contractor might 
receive several color classifications under one contract. For 
example, he might be classified Red in quality, Green in cost 
control, and Blue in the remainder. 

When making assessments of contractors, the AFSC 
Regulation covering CPARS cautions program managers to base 
their evaluations on facts. Their assessments should be 
Supported by contract management data "such as_ cost 
performance reports, technical interchange meetings, financial 
solvency assessments, production management reviews, 
Goneractor operations reviews, functional performance 
evaluations, and earned contract incentives." [Ref. 13:p. 1] 
The Regulation specifically states, "Subjective assessments 
concerning the causes or ramifications of the contractor's 
performance should be provided; however, speculation or 
conjecture should not be included." iiket et] oem 

The intention of the program is that during source 
evaluation, CPARsS will be used to evaluate a contractor's past 
performance and assess the risk involved in doing business for 
the contract under consideration. Specifically, during the 
source selection process, the contracting officer will check 
the CPARS of contractors submitting proposals. If he 
determines that the effort described in the CPARS applies to 
the current procurement, they will be assembled into a report 
for the Performance Risk Analysis Group (PRAG). The PRAG 


considers the CPARS along with other contractor past 
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performance data to determine contractor risk. The PRAG 
usually expresses contractor risk as low, medium, or high. 
This risk assessment is then used as part of the source 
selection. [Ref. 14] 


At present, each Product Division within the AFSC 


Maintains a complete set of hard copy CPARs. The Air Force 
intends eventually to automate CPARS. The automation effort 
has been delayed for two reasons. First, the resources 


required to input the CPARs and ensure compatibility with the 
computer hardware at all commands are not currently available. 
Second, the Air Force is still developing a method to ensure 
the security of the automated data. [Ref. 14] 

Each CPAR will be retained for five years unless the 
program manager requests a longer retention period. CPARSs are 
considered source selection sensitive because they will be 
used for procurements throughout their retention period. They 
may also contain proprietary information. Therefore, all 
CPARS and attachments will be marked "For Official Use Only/ 
Source Selection Sensitive." Additionally, the forms and the 
data base (if automated) must be protected from disclosure to 
organizations or personnel not involved in the source 


selection process. [Ref. 13:p. 3] 


. BLUE RIBBON CONTRACTOR PROGRAM 
The Blue Ribbon Contractor program at NAC Indianapolis 


began in December 1987. It was inspired by the Packard 
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Commission recommendations of 1986 [Ref. 16] and the Blue 
Ribbon Contractor program at Tinker Air Force Base begun a 
year earlier [Ref. 17]. NAC has attempted to learn from some 
of the difficulties experienced at Tinker and has tailored the 
program to respond to their concerns and conform to their 
resource constraints [Ref. 18]. 

The program at NAC is applied to contracts, purchase 
orders, and blanket purchase agreements with up to a $25,000 
estimated value [Ref. 16:p. 41]. Procurements made for the 
bomb-rack program, which have historically caused difficulties 
for NAC, are excepted from this dollar limit and are all 
included in the Blue Ribbon Contractor program [Ref. 19]. 
Other exemptions to the program are procurements awarded via 
sealed bidding procedures and involving small, disadvantaged 
business set-aside awards. Additionally, the program is not 
a substitute for procedures governing contractor respon- 
Sibility determination [Ref. 19:p. 2]. 

In order to qualify as a Blue Ribbon contractor, a 
contractor must first apply in writing to NAC. If he has made 
one delivery during the last 12 months, he will be placed on 
the list in that category. To receive the benefits of being 
designated a Blue Ribbon contractor, however, he must have and 
maintain an on-time delivery rate of 95 percent or greater. 
(Ref. 19:p. 2] This delivery rate is calculated by dividing 
the number of delinguent orders by the number of open orders 


and multiplying by 100 [Ref. 18]. Only data collected at NAC 
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Indianapolis are used in determining this delivery rate [Ref. 
mop. 3). fies hues kiebon Contractor does not qualify to 
receive the benefits of Blue Ribbon status upon application, 
or if his on-time delivery rate drops below 95 percent, he is 
notified that he is still listed as a Blue Ribbon contractor, 
but he cannot exercise the benefits of the program until his 
delivery rate improves [Ref. 18]. 

There are three benefits accruing to Blue Ribbon 
contractors with a qualifying delivery rate. PiLtse, “ene 
contracting officer has the authority to award to a Blue 
Ribbon contractor over other offerors at a price up to ten 
percent higher than the lowest priced responsible offeror. If 
the procurement iS in support of the Navy bomb rack progran, 
the contracting officer has the authority to award to a Blue 
Ribbon contractor at a price up to 20 percent higher than the 
lowest priced responsible offeror. Re tee open | Second, 
NAC Indianapolis solicits Blue Ribbon contractors to the 
maximum extent possible. The bidders list is rotated, but 70 
percent of it is made up of Blue Ribbon contractors. (Ref. 
18) Third, Blue Ribbon contractors are permitted to use their 
status in their marketing programs [Ref. 18]. 

So far, the Blue Ribbon Contractor program has resulted in 
only 3.5 percent of awards (of contracts in the program) being 
made to Blue Ribbon contractors that were not the lowest 
priced responsible offerors. However, the delinquency rate on 


blanket purchase agreements has dropped from 67 percent before 
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the start of the program to 8.7 percent in July 1989. Seventy 
percent of the suppliers awarded contracts in July 1989 are 
the same suppliers that were receiving awards when the program 
began. One protest has been filed by a low offerer who lost 
a contract to a higher priced Blue Ribbon contractor. The 
General Accounting Office dismissed the protest because it was 
not timely. [Ref. 18] Therefore, no protest related to the 
Blue Ribbon Contractor program has yet tested the program's 
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D. TOTAL QUALITY MANAGEMENT 

TOM iS a management style introduced to the Japanese in 
the late 1940s by Americans participating in the post-World 
War II effort to rebuild Japan. TQM proved to be a very 
successful management method in Japan. As a result, it is 
gaining an increasing following within U.S. industry trying 
to regain competitive position both internationally and 
domestically. Based on successes in Japan and within the 
U.S., DOD is attempting to adopt TQM in an effort to operate 
more efficiently in a time of dwindling resources. This 
section of Chapter III will attempt to describe the TQM effort 
as it applies to DOD and, more specifically, DOD procurement. 
First, it will describe TQM in general, concentrating the 
discussion on those aspects of TQM that most directly impact 


upon procurement practices and, therefore, that would compete 
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with RYG. Second, it will outline current DOD implementation 
plans related to acquisition. 
1. TOM Implementation--General 

Adopting TQM means~ that everyone within an 
organization must be involved in improving the quality of 
fear product. Whether they are manufacturing a product or 
preparing documents, everyone produces something and every 
product has a customer. Therefore, everyone in the 
organization can improve quality. The basic theory behind TQM 
is that as quality improves, productivity improves because 
rework is reduced [Ref. 20]. As quality and psOducEIVIEY 
rise, competitive position also improves [Ref. 21]. 

One of the key figures in the introduction of TQM to 
the Japanese is Dr. W. Edwards Deming. Dr. Deming has devised 
14 principles for management which he believes U.S. firms must 
follow if they desire to adopt TQM. His 14 principles are: 

1. Create constancy of purpose toward improvement of 
product and service, with the aim to become competitive 
and stay in business, and to provide jobs. 

2. Adopt the new philosophy. We are in anew economic age. 
Western management must awaken to the challenge, must 
learn their responsibilities, and take on leadership for 
change. 

3. Cease dependence on mass inspection to achieve quality. 
Eliminate the need for inspection on a mass basis by 
building quality into the product in the first place. 

4. End the practice of awarding business on the basis of 
price tag. Instead, minimize total cost. Move toward 


a Single supplier for any one item, on a long-term 
relationship of loyalty and trust. 


Sus. 


5. Improve constantly and forever the system of production 
and service, to improve quality and productivity, and 
thus constantly decrease costs. 


6. Institute training on the job. 


7. Institute leadership (see Point 12). The aim of super- 
vision should be to help people and machines and gadgets 
to do a better job. Supervision of management is in 
need of overhaul, as well as supervision of production 
workers. 


8. Drive out fear, so that everyone may work effectively 
for the company. 


9. Break down barriers between departments. People in 
research, design, sales, and production must work as a 
team, to foresee problems of production and in use that 
may be encountered with the product or service. 


10. Eliminate slogans, exhortations, and targets for the 
work force asking for zero defects and new levels of 
productivity. Such exhortations only create adversarial 
relationships, as the bulk of the causes of low quality 
and low productivity belong to the system and thus lie 
beyond the power of the work force. 


11. Eliminate work standards (quotas) on the factory floor. 
Substitute leadership. Eliminate management’ by 
objective. Eliminate management by numbers, numerical 


goals. Substitute leadership. 


12. Remove barriers that rob the hourly worker of his right 
to pride of workmanship. The responsibility of super 
visors must be changed from sheer numbers to quality. 
Remove barriers that rob people in management and in 
engineering of their right to pride of workmanship. 
This means, inter alia, abolishment of the annual or 
merit rating and of management by objective. 


13. Institute a vigorous program of education and self- 
improvement. 


14. Put everybody in the company to work to accomplish the 
transformation. The transformation is everybody's job. 
fPREE ~ 2Os pp. 28-24u 

Although all 14 points are interrelated, two have the 


greatest implications for changing traditional procurement 
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Practices. These are number 3, "Cease dependence on mass 
inspection" and number 4, "End the practice of awarding 
business on the basis of price tag." Therefore, the 
discussion in this section will focus on how these two 
principles should be implemented. 

The push to do away with mass inspection is based on 
the theory that inspection is ineffective in ensuring quality. 
This is true for two reasons. First, inspection occurs too 
late in the process. If quality has not been built in during 
production, it cannot be inspected in [(Ref. 20:pp. 28-29]. 
Inspection can only separate good items from bad. Second, 
even if 100 percent inspection is employed to separate good 
from bad, some defective material will slip through. One 
hundred percent inspection is unreliable because inspectors 
are human and make errors, no matter how conscientious, 
because of boredom and fatigue. [Ref. 20:p. 29] In addition 
to being ineffective, mass inspection iS very expensive. 
First, it 1s very labor intensive, so it requires the 
employment of many inspectors. Second, the items that fail 
inspection are either discarded or reworked. The first 
alternative wastes the costs of material and production used 
to make the item. The second alternative wastes’ the 
additional material and man hours employed in reworking the 
item. These are all resources that could be better employed 


elsewhere. 
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In place of mass inspection, Statistical Process 
Control (SPC) should be employed. SPC is a method to build 
quality into a product during preduction] It as 4 tooleres.. 
used by management to identify the reasons a product is not 
produced with consistency. Management charts a process, 
perhaps one segment of production, to see the fluctuations in 
the output of the process. If the process exhibits more than 
Simple random variation, it is not in statistical control. 
This means that local sources of trouble are affecting the 
process. fRet. S245 p..Se Examples of local problems are 
differences in materials used in production, readjustment of 
machinery at the beginning of each shift, and use of different 
procedures on each shift. These local problems must be 
eliminated to bring the process into statistical control. 
Only when the process is in control, can improvements be made 
effectively. This 1S because if the process is not in 
control, management will not be able to observe the affects of 
changes made to it. [Ref. 21:p. 5] Once the process is in 
control, changes made to it will shift the entire control 
chart in either a favorable or unfavorable direction, giving 
management feedback about its innovations. Management can now 
follow the Plan, Do, Check, Act (PDCA) cycle. First, data are 
analyzed to determine what change might improve the process. 
Second, management tests the idea by putting it into practice. 
Third, the control charts are monitored to determine the 


affect (favorable or unfavorable) of the change. Finally, 
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based on the feedback from the third step, the change is 
either implemented permanently or discarded. The PDCA cycle 
is then begun again. The result is continuous improvement of 
the process in smaller and smaller increments. 

The key to employing sampling inspections and SPC in 
place of mass inspection is the determination of which 
characteristics to monitor and which inspection method to use. 
Dr. Ishikawa explains that it is important to inspect quality 
characteristics that are unstable, thus causing defects, as 
well as vital characteristics that affect the life of a 
product [Ref. 22]. These inspections are not performed after 
the product is complete; they are conducted by the worker 
during the production process. In addition, only a sample of 
the items produced are inspected to either determine whether 
the process is in control or to be sure that the process 
remains in control. Because only a small sample of items are 
inspected, inspectors can do the job properly. [Ref. 20:pp. 
29-30] Dr. Ishikawa further explains that the goal of SPC 
should be a product with 100 per cent acceptable quality. SPC 
makes this goal feasible. PRGtawweeceopp. ~Ligy=i1718 } Mass 
inspection, on the other hand, is guaranteed to result in some 
percentage of failures. 

The second principle that applies directly to 
procurement is number 4, "End the practice of awarding 
business on the basis of price tag." What this principle 


means is that when purchasing material, an organization should 
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consider the lowest total cost of the item rather than the 
lowest initial cost [Ref. 23]. The organization must have 
long term goals. Buyers have a new job. They have 
traditionally sought lower and lower prices for material with 
little or no concern for quality or service. [Ref. 21:p. 25] 
It is difficult to select a contractor with a higher initial 
price since it can be very subjective when defending the 
selection during an award protest. Now buyers must 
concentrate on quality because, as Dr. Deming states: 


Price has no meaning without a measure of the quality being 


purchased. Without adequate measures of quality, business 
drifts to the lowest bidder, low quality and high cost being 
the inevitable result. American industry and the U.S. 


Government, civil and military, are being rooked by rules 
that award business to the lowest bidder. [{Ref. 23:p. 23] 


In order to ensure receipt of quality goods and 
services, an organization must do two things. First, it must 
require that its suppliers practice TQM. Second, it nmuee 
reduce the number of suppliers it deals with and seek long- 
term relationships with these suppliers. 

To verify that a supplier practices TQM, an 
organization should require the supplier to provide 
statistical evidence of quality. This means supplying the 
buying organization with control charts and proof that the 
Supplier is working on changing its management style to 
conform with Deming's principles. (Ref. 9215p oes The 
buying organization must be sure that the control charts 


provided represent the proper inspection technique taken at 


40 


Sieewcritical points in the production process using the 
correct sample size. This requires a close relationship 
between the buying activity and the company supplying the 
material. The activity buying material must be sure that the 
Supplier is capable of producing a quality product before the 
item is produced. In Dr. Deming's words, "The customer that 
waits for delivery of material to learn what he has bought 
will take what he gets." [Ref. 20:pp. 40-41] This concept 
opens a new world of responsibilities to the contract 
administration activities within DOD. It suggests that their 
traditional role inspecting the quality of material GQbeere) & 
has been produced must change. Instead, they would be 
expected to participate with the contractor in verifying the 
quality of his production processes before and during 
production of required material. 

The concentration on long-term goals and close buyer- 
supplier relationship requires a long-term relationship with 
weemall number of Suppliers. This is in direct conflict with 
the Competition in Contracting Act of 1984 and a concept that 
DOD must deal with to successfully implement a TQM philosophy. 
The buying activity's goal is still to purchase material of 
high quality at the least cost. However, the buying activity 
must also devote resources to assist the vendor in employing 
TOM if it is to succeed. Therefore, as the number of 
suppliers an activity deals with increases, the number of 


resources devoted to ensuring a TQM philosophy also increases. 
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Even after the activity is working with suppliers dedicated to 
TOM and is receiving high quality material, if this material 
is supplied by two different vendors, there will be 
differences. These differences will surface when the material 
is placed into the production process. {[{Ref. 20:p. 35] For 
these reasons, Dr. Deming encourages organizations to reduce 
their suppliers for a given end item to one [{Ref. 20, p. 23]. 
Ishikawa believes, however, that an organization should 
maintain at least two suppliers for a given product in case 
one firm is unable to supply the item [Ref. 24]. 

A supplier needs the promise of a long-term contract 
to make it cost effective for him to institute TQM and 
concentrate on continuously improving his production process 
[R@fiicmee2 12. Sa Without some guarantee of continued 
business, he will not be willing to invest the resources 
necessary to bring his processes into control and then improve 
upon them. DOD must seriously consider the awarding of multi- 
year contracts (two to five years) or contracts with optional 
years to assure a contractor that DOD is supporting a basic 
TQM mandate. Contractors are currently forced to concentrate 
on making a profit in the short term since DOD is focused on 
the short term. If, however, a long-term relationship can be 
developed, both customer and supplier will benefit. The 
customer will receive high quality material at lower costs. 
The supplier will no longer need to employ resources in 


storing and reworking defective material. [Ref. 21:p. 115] 
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2. TOM Implementation--DOD 

The application of TQM within the Federal Government 
began in 1986 when President Reagan signed Executive Order 
12552. The goal of this Executive Order is to significantly 
improve the productivity of the Federal Government by 1992 
through employment of TQM. [Ref. 25} In 1988, DOD initiated 
its TQM program, issuing the Total Quality Management Master 
Plan in August of that year. 

The strategy of the Master Plan "aims at achieving one 
broad, unending objective: continuous improvement of products 
and services." [Ref. 26] In order to achieve this objective, 
the plan includes goals for the short term (one year), mid 
term (three years), and long term (seven years). These goals 
cover a broad range of DOD tasks. They state, however, that 
the acquisition community will be the first to implement TQM 
and that others will follow in approximately three years [Ref. 
weep. 3). Since the acquisition community is leading TQM 
implementation and this thesis will concentrate its discussion 
of TOM in the areas relating to RYG, discussion of the Master 
Plan will be limited to procurement and Government-industry 
relationships. 

In the short term, two of the seven goals directly 
impact procurement. The first of the two is to "Begin 
enlisting Defense industry commitment." This goal intends to 
begin TQM implementation by using programs already in 


existence within DOD. These programs are to be used to 
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communicate to industry DOD's desire for continuous process 
improvement. (Ref. 26:p. 10] The second applicable goal is 
to "Develop and implement recognition and reward system based 
on TQM goals and behaviors." This goal discusses the need to 
revise existing performance incentives to encourage and reward 
continuous improvement efforts. [Ref. 26:p. 10] 

One of the nine mid-term goals applies directly to 
procurement: "Implementation commitment by major Defense 
contractors, with 'critical mass' achieved in at least the top 
25 contractors." This goal intends to result in a commitment 
from the top 25 Defense contractors to independently implement 
continuous process improvement programs. These 25 contractors 
receive contracts valuing more than 50 percent of DOD's total 
acquisition expenditures, and they receive most of DOD major 
system contracts. This commitment is expected to involve the 
training of senior leadership within these firms in TQM and 
their active involvement in continuous improvement efforts. 
[Rete UG. po! 

In the long term, one of the four goals of the Master 
Plan impacts upon procurement: "Widespread Defense industry 
implementation of continuous process improvement." 
Implementation of this goal involves employment of TQM by all 
DOD prime and subcontractors. Its achievement requires three 
things .ofs DOD. First, DOD must encourage and assist 
contractors in their implementation of continuous process 


improvement. Second, DOD is expected to develop criteria to 
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evaluate contractor commitment to continuous improvement. 
Finally, the degree and success of contractor process 
improvement efforts are to be included in source selection 
strategies. [{Ref. 26:p. 4} Within this goal, the Master Plan 
states, "The intent of continuous process improvement efforts 
is to recognize and reward dedication and adherence to the 
improvement process, and not restrict reward to attaining a 


specific standard of performance." [Ref. 16:p. 4] 


E. SUMMARY 

This chapter briefly described three DOD contractor 
quality programs. The programs described were the Air Force 
CPARS, NAC Indianapolis Blue Ribbon Contractor program, and 
powe TOM initiative. The procedures associated with each 
program have been provided as background information to 
Support the next chapter. The chapter that follows will 
discuss the merits and shortcomings of the three programs and 


their relationship to RYG. 
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IV. A COMPARISON OF RED YELLOW GREEN AND OTHER 
CONTRACTOR QUALITY INITIATIVES 
Rs INTRODUCTION 

This chapter will use the background information provided 
in Chapter III to analyze the Red Yellow Green (RYG) program 
relative to the other three programs described--the Contractor 
Performance Assessment Reporting System (CPARS), the Blue 
Ribbon Contractor program, and the Total Quality Management 
(TQM) initiative. The purpose of the analysis is to determine 
whether RYG should co-exist with these other initiatives or if 
it should replace or be replaced by any one of them. 

The assessment will be organized in the same way as the 
background information presented in Chapter III. First, RYG 
will be compared to CPARS. Next, RYG will be evaluated 
relative to Naval Avionics Center (NAC) Indianapolis' Blue 
Ribbon Contractor program. Finally, RYG will be assessed for 
its compatibility with the Department of Defense (DOD) TOM 


inteclatave. 


Bs CONTRACTOR PERFORMANCE ASSESSMENT REPORTING SYSTEM 

The CPARS devised by the Air Force is intended for use in 
source selections for major systems contracts. Therefore, the 
threshold for employment of CPARS is set at $5 million and the 
number of contractors included in the program has been limited 


to 84. The Air Force had two major reasons for setting these 
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mimics. The first involves maintenance of the Contractor 
Performance Assessment Reports (CPARs). The second involves 
the nature of the information reported in the CPARSsS and, 
therefore, the way in which this information must be used in 
the source selection process. These two factors will be 
discussed in detail in the following paragraphs. 

Maintenance of CPARS requires many resources. First, it 
involves the time invested by the program manager to prepare 
the report. Although the reports are supposed to be based on 
facts, they are not merely collections of data. They are 
written, subjective assessments of the contractor's perfor- 
mance and, therefore, require some effort on the part of the 
program manager. The second factor concerning maintenance of 
CPARsS involves the efforts of those who verify the information 
contained in the reports, ensure that the reports are current, 
and physically store the reports. Since the CPARS are 
expected to be used in future source selections, the Air Force 
carefully verifies the information contained in the reports to 
ensure that it 1S compatible with past trends and other CPARs 
filed on the same contractor. When an activity uses CPARS in 
source selection, the currency of the information is checked. 
If a more current CPAR is desired, the procuring activity 
contacts the program office and requests more up-to-date 
mreormation. In addition, since the system 1S not yet 
automated, maintaining hard copy files of all CPARs at the six 


Product Divisions within the Air Force Systems Command 
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requires a tremendous clerical effort. Maintenance of the 
CPARsS is a very labor intensive process. 

The information contained in CPARS is subjective. It is 
based on facts, but much of it is still the opinion of the 
program manager and members of the program office. 
Additionally, because a CPAR is a written report, it is a 
relatively long document. Because of its subjectivity and 
length, the use of CPARs in source selection requires careful 
evaluation of the information contained in the report. TS 
perform this evaluation, the CPARs must first be collected, 
evaluated for their relevance to the procurement in question, 
and assembled into an integrated report. The resulting report 
is then further evaluated in conjunction with other past 
performance data to determine a contractor's level of risk. 
This is a relatively long process. The Air Force believes the 
process is beneficial for major weapon system procurements. 
However, CPARS is not used for non-major acquisitions. 

The points discussed above suggest that CPARS would not be 
appropriate for contracting operations at field activities for 
three reasons. 

First, the investment in resources to maintain the CPARs 
could not be justified by the relatively small dollar 
procurements awarded by field contracting activities. The 
return on investment to obtain higher quality material would 
surely be less than the cost to prepare and maintain the 


CPARs. 
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Pecomd, Im Order to be effective in field contracting, the 
number of contractors included in the program would have to be 
greatly expanded. The contractors that compete in major 
system procurements are relatively few. However, field 
contracting offices deal with thousands of contractors. Even 
if fewer than 100 contractors were included in the system, 
Prema activities would find it very difficult to obtain 
accurate past performance information. An expansion of CPARS 
at the field level to accommodate such a large number of 
contractors would be an impossible undertaking. 

Finally, even if CPARS were expanded, field contracting 
offices would have neither the time nor the resources to 
perform the evaluation necessary to correctly apply it to 
source selection. 

In summary, CPARS seems to be only appropriate for use in 
the procurement of major systems. It should not be considered 
for expanded use in field contracting. RYG was designed for 
use by field contracting offices. Accordingly, its formula 
driven assessment of past performance would not be appropriate 
for use in major system procurements. The two programs were 
designed to satisfy different needs in different areas of 
Government contracting. Therefore, neither should be 


considered as a replacement for the other. 
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er BLUE RIBBON CONTRACTOR PROGRAM 

The Blue Ribbon Contractor program at NAC Indianapolis has 
five unique features. First, it is an automated system 
created by NAC using data collected by NAC. Second, the 
criteria used to define a Blue Ribbon contractor and the 
programs included in the program were chosen to focus 
attention on the critical and problem materials procured by 
NAC. Third, contractors must apply for Blue Ribbon status to 
be included in the program, and NAC performs the analysis to 
determine whether these contractors qualify for the benefits 
of the program. Fourth, the program concentrates on 
commodities important to NAC's mission, and a contractor must 
make a delivery to NAC at least once each year to be included 
in the program. Therefore, the contractors designated as Blue 
Ribbon will generally be the contractors NAC deals with on a 
regular basis. Fifth, use of the program in source selection 
requires judgment. Unlike RYG, it is not a formula-driven 
program. The decision to award to a Blue Ribbon contractor 
over a lower-priced offeror (not included in the program) 
requires an assessment by both contracting and quality 
assurance personnel. Therefore, the two groups must 
communicate with each other to use the program effectively. 

Like RYG, the Blue Ribbon Contractor program iS ap- 
propriate for use in field contracting. However, a Blue 
Ribbon program requires that an activity devote many more 


resources than does RYG. First, the activity would need the 
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capability to automate. Second, it would have to devise the 
criteria to be used in designating Blue Ribbon contractors. 
NAC chose delivery, but other commands might be more 
interested in quality or reliability. Third, it would have to 
decide which commodities are important enough to its mission 
to justify inclusion in the program. Fourth, it would need to 
devote resources to the program to evaluate applicants and 
maintain the Blue Ribbon Contractor list. Finally, the 
contracting and quality assurance personnel in the command 
would have to be willing and able to work together to 
implement the program. 

Some commands would be unable to take the five steps 
discussed in the previous paragraph. They might lack the 
automation necessary to implement the program. Or, 
contracting personnel might be physically located away from 
quality assurance personnel. This would be the case for a 
Navy Regional Contracting Center procuring goods and services 
for many of its Navy field activities. In either case, 
implementation of a Blue Ribbon Contractor program would be 
impractical. Therefore, a Blue Ribbon program does not seem 
to be a viable option for every field contracting activity. 

The question then is whether RYG and Blue Ribbon 
Contractor programs could co-exist at activities capable of 
implementing both programs. This researcher believes the 
answer to that question is yes--the programs are compatible 


and can enhance each other. If used together, the Blue Ribbon 
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program would be used when procuring goods and services that 
the command considers critical to the performance of its 
mission. The Blue Ribbon program would allow the command to 
concentrate its attention on those parameters that it 
considers important. RYG could be used for all other procure- 
ments. By definition, RYG procurements would be for less 
critical and less frequently purchased materials. These items 
would not justify the resources needed to apply the Blue 
Ribbon program. In addition, the command may not have been 
able to generate enough local data on critical items to assess 
contractor past performance. RYG would allow the command to 
consider past performance in source selection while employing 
a very simple system. 

Selected field activities could benefit from using both 
RYG and a tailored Blue Ribbon Contractor program. The most 
likely evolution would be to first implement RYG since it is 
less complex. Use of RYG should help personnel involved in 
the source selection process become comfortable with using an 
automated system. In addition, implementation of RYG should 
force contracting and quality assurance personnel to work 
together. RYG requires quality assurance personnel to use 
past performance to evaluate offerors and determine the 
quality assurance actions required of each. They must then 
provide this assessment to contracting. Therefore, RYG 
involves quality assurance more deeply in the- source 


evaluation process. Both will help pave the way for 
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iiiemoduction Of a Blue Ribbon Contractor program concentrating 
attention on the area of the command's specialization. [Ref. 


27) 


1 TOTAL QUALITY MANAGEMENT 
The discussion in this section will focus on whether RYG 
furthers the goals of TQM. The purpose of this discussion is 
to evaluate whether RYG 1S consistent with DOD's TOM 
initiative or whether RYG hinders this initiative. This 
section will first consider RYG's conformance to the general 
goals of TQM as outlined in Chapter III. . Second, it will 
relate RYG to the goals in the DOD TQM Master Plan. 
1. TOM and RYG--General 
Two of Dr. Deming's 14 principles were determined to 
directly impact the Government procurement process. and, 
therefore, apply to the comparison of TQM and RYG. These two 


principles are: 


1. Number 3. "Cease dependence on masS inspection to 
achieve quality. Eliminate the need for inspection on 
amass basis by building quality into the product in the 
Mmitatenplace. "  e(Ref. 20: p.023 |] 

2. Number 4. "End the practice of awarding business on the 


basis of price tag. Instead, minimize total cost. Move 
toward a Single supplier for any one item, on a long- 
term relationship of loyalty and trust." [Ref. 220°). 
pe) 
The first concept addresses the requirement for contractors to 
practice statistical process control (SPC). The second 


addresses the relationship between buyers and suppliers. 
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Implementation of it involves considering total cost when 
awarding contracts by ensuring that suppliers practice TOM. 

RYG does not satisfy all of the requirements of these 
two principles, nor does it require contractors to practice 
all of Deming's 14 fprimeiples- It does, however, help the 
Government end the practice of awarding business on the basis 
of price tag by considering the total cost of a procurement 
when the cost of poor quality is added to the source selection 
formula. Therefore, in a limited way, RYG helps to further 
the goals of TQM. 

This researcher believes that a limited implementation 
of TOM in field contracting 1S probably a more realistic 
undertaking for DOD. Even the third and fourth Deming 
principles will be very difficult to implement since they 
require two resources that may not be available to field 
contracting officers--time and personnel. 

Time is required to ensure that the supplier practices 
TOME Contracting personnel would have to examine the TQM 
programs of offerors in detail. They would need to ensure 
that the contractors practice SPC as well as other Deming 
principles. Implementation of the management principles is 
more difficult and more time consuming because it involves 
evaluating the firm's daily management’ style. Field 
contracting requires the processing of many procurement 


actions ina relatively short period of time. Contracerma 
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offices do not devote the time necessary to perform such an 
on-going evaluation. 

In addition to time, personnel are required to perform 
a vendor TQM assessment. As discussed in Chapter III, the 
buying activity must be able to evaluate whether the supplier 
is taking the correct measurements at the critical points in 
production and if the sampling technique is proper. This 
requires an in depth knowledge of both the production process 
and statistics. Someone with technical knowledge of the item 
being purchased, perhaps an engineer, would have to be a 
permanent member of the evaluation team. Additionally, a 
statistician would probably be needed. It is unlikely that a 
contracting office would have either the number of personnel 
or skill levels required for the task. 

2. TOM and RYG--DOD 

With the exception of its short-term goals, the 
segments of the DOD TQM Master Plan relating to procurement 
seem to focus on procurement of major systems. Thrs 1s 
probably due to the dollar values involved in major system 
procurements as well as the degree of management attention 
focused on each program. The expected returns from using TOM 
for major system acquisitions would be greater than for 
smaller procurements. In addition, the time schedule of a 
major system acquisition would more easily allow the 
contractor evaluation required by TQM. Procurements made in 


the field do not involve such high dollar values. Neither do 
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they allow the degree of management attention required by 
procurement of a major system. However, in spite of the fact 
that the Master Plan does not specifically address contractor 
quality programs intended for field contracting, thm 
researcher believes that RYG can still be shown to conform to 
the intent of the Master Plan. 

In the short term, the Master Plan discusses enlisting 
defense industry commitment to TQM by using programs already 
in existence within DOD. Additionally, it hopes to employ 
recognition and reward systems based on TQM goals’ and 
behaviors. [Ref. 26:p. 10] RYG is consistent with both these 
objectives. 

First, RYG employs an existing system, the Product 
Deficiency Reporting and Evaluation Program (PDREP), to 
improve the quality of material received by the Navy. It 
expands the application of this system to allow greater 
consideration of contractor past performance in the source 
selection process. Second, RYG could be used to reward 
contractors who provide good quality material to the 
Government. It accomplishes this by considering the cost of 
poor quality when making source selection decisions. Since 
proposals from contractors with good quality history are not 
assessed a Technical Evaluation Adjustment, these contractors 
have a competitive advantage over those with a poor quality 


history. 
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The Master Plan's procurement goal for the mid term 
addresses only major system procurements. It involves getting 
a commitment from DOD's top 25 contractors to implement TOM. 
These contractors receive more than 50 percent of DOD's 
procurement dollars and most Of its major system 
contracts. (Ref. 26:p. 6} This goal is not related to RYG 
Since RYG neither helps nor hinders its implementation. 

In the long term, the Master Plan seeks employment of 
[elieoy all DOD prime and subcontractors [Ref. 26:p. 4]. As 
discussed earlier in this section, it is difficult for field 
contracting offices to require and verify employment of TOM by 
the vast number of contractors dealt with at this level. 
However, RYG at least allows them to consider improved 
quality, which is the result of continuous’ process 
improvement, in their source selection decisions. Therefore, 
use of RYG is at least a step toward TQM implementation in 
field contracting. Additionally, implementing a Blue Ribbon 
Contractor program in conjunction with RYG can move field 
contracting activities even closer to the goals of TQM. 

As described in the Blue Ribbon Contractor Program 
section of this chapter, the RYG and Blue Ribbon Contractor 
programs can enhance each other. RYG 1S necessary for 
procurement of infrequently used and non-critical materials 
that do not justify a great deal of management attention. It 
is also needed by commands incapable of implementing a Blue 


Pabsioem) Contractor program. RYG focuses on the negative 
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aspects of poor performance to improve quality. A Blue Ribbon 
Contractor program requires more resources to implement than 
does RYG and so would be used in the procurement of materials 
an activity considers critical. Blue Ribbon programs focus on 
the positive aspects of past performance to improve quality. 

Because of its simplicity and focus on the negative, 
RYG could be considered a first step toward implementing TQM 
in field contracting. The program will always be needed when 
a Blue Ribbon program is either unavailable or inappropriate. 
Since the Blue Ribbon Contractor programs are more complex and 
focus on the positive, they are a logical second step toward 


implementing TQM in field contracting. 


ie SUMMARY 

This chapter evaluated the three DOD quality programs 
described in the previous chapter--CPARS, the Blue Ribbon 
Contractor program, and the TQM initiative. It compared and 
contrasted these programs to RYG and discussed whether RYG 
should co-exist with them. The next chapter will assess the 
strengths and weaknesses of the RYG program. The analysis 
will include a discussion concerning the program's ability to 
withstand a protest. It will also assess the current status 


of the RYG test. 
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V. STRENGTHS AND WEAKNESSES OF RED YELLOW GREEN 


A. INTRODUCTION 

This chapter will attempt to assess the strengths and 
weaknesses of the Red Yellow Green (RYG) program. First, the 
four areas of the program most open to contractor protests 
will be addressed. The first area considered will be whether 
RYG results in de facto debarment of Red and Yellow contrac- 
tors. The second consideration is whether the Technical 
Evaluation Adjustments (TEAS) could be regarded aS penalties 
and whether it is fair for the Government to consider costs of 
quality (a basic premise of RYG) in the source selection 
decision in the first place. The third to be discussed will 
be whether consideration of past performance, as an evaluation 
factor, constitutes a responsibility determination. The 
fourth area addressed in this chapter will be the accuracy of 
the data base used to rate contractors Red, Yellow, or Green. 
The overall discussion will include an evaluation of the 
program's ability to withstand a contractor protest. 

Next, the progress of the RYG program test will be 
examined. This discussion will first involve an assessment of 
test implementation at each of the five test activities. It 
will then provide data documenting the results of the test to 


date and discuss the significance of these data. 


oye, 


Be PROTEST ISSUES 
1. De Facto Debarment 
The discussion concerning de facto debarment will 
center around whether designating a contractor as Red or 
Yellow has the same effect as debarment. The effect of 
debarment as described in the Federal Acquisition Regulation 
(FAR) is: 

(a) Debarred or suspended contractors are excluded from 
receiving contracts, and agencies shall not solicit offers 
from, award contracts to, or consent to subcontracts with 
these contractors, unless the acquiring agency's head or a 
designee determines that there is a compelling reason for 
SUCH ACC LON eivicceen | RO EEECION 

Cibinic and Nash provide a similar description of the effect 
of debarment: 

The second broad sanction is debarment of the contractor 
or its employees from taking any Government contracts for a 
stated period of time....Debarment precludes contracts with 
any agency of the federal government for a period of up to 
three years.... (Ref. 29] 

Based on the above definition, it is clear that debarment of 
contractors excludes them from competing for any Government 
COmMemace. This means that the Government must avoid both 
soliciting offers from and awarding contracts to debarred 
CONneraccers. 

Application of the RYG test procedures neither stops 
the Government from soliciting offers from nor awarding 
contracts to Red and Yellow contractors [Ref. 30]. It merely 


allows the Government to consider the costs it expects to 


incur based on the quality of material it is most likely to 
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meceive from an offeror. If, after considering the costs 
related to additional quality assurance requirements, a Red or 
Yellow contractor is the lowest-priced responsible offeror, 
the Red or Yellow contractor will be awarded the contract. 

Since the Government will be soliciting proposals 
from, and awarding contracts to, Red and Yellow contractors, 
application of the RYG test procedures is not considered a de 
facto debarment in the analysis of the researcher. It is 
expected that the RYG program will be able to withstand any 
protest in this area. 

2. Use of TEAS 

The second area for potential protest is the use of 
TEAS. The propriety of using TEAs will be considered first 
by discussing whether TEAS could be considered penalties and, 
therefore, unenforceable, and second by discussing whether it 
is fair for the Government to consider costs of quality when 
evaluating proposals. In order to assess the reasonableness 
of the values assigned as TEAs, the researcher considered 
precedent associated with the reasonableness of liquidated 
damages amounts. 

Regulations and precedent applying to liquidated 
damages were used to determine whether the TEAS (as calculated 
in the RYG test procedures) could be considered penalties. 
The FAR states that "liquidated damages fixed without any 
reference to probable actual damages may be held to be a 


penalty, and therefore unenforceable." [Ref. 28:para. 12.202] 
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Based on this regulation, the question then becomes whether 
the TEAs are related to actual costs that the Government would 
expect to incur when forced to carry out additional quality 
assurance requirements. 

Cibinic and Nash state, "The Government contract rule 
is that the reasonableness of the forecast of liquidated 
damages is evaluated by looking at the situation at the time 
the contract 1S) made. a. [Ret . 29° Dp. coe They further 
explain that even liquidated damages calculated at a fixed 
rate of dollars per day have been upheld by the courts because 
they are considered to cover the Government's administrative 
expenses, "i.e., inspection, superintendence or engineering 
costs, which the Government will incur if a delay is 
encountered." [REE 3929-0. 7603) In the case of Young 
Associates, Inc. v. United States, 200 Ct. Cl. 438,471 F.2d 
618 (1973) the court stated: 


The answer is, we think, that the regulation does not 
require a liquidated-damage schedule to be tailor made for 


each individual contract. It 1S enough if the amount 
stipulated is reasonable for the particular agreement at the 
time it is made. [Ref. 29:p. 804] 

In summary, amounts of liquidated damages are 


considered reasonable if they are a fair estimate (calculated 
when the contract is made) of the administrative costs the 
Government would expect to incur during a delay. Further, the 
estimate need not be tailored to each contract. Based on 
this, amounts of TEAs should be considered reasonable if they 


are truly a fair estimate of the administrative costs the 
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Government expects to incur because a contractor has poor 
Suabaty history. 

The TEAS furnished as examples in Chapter II and 
Appendix D were calculated using Navy Material Quality 
Assessment Office (NMQAO) historical data. These data include 
labor hours required for the quality assurance actions, labor 
rates, and costs of testing. PReft. 2) They are, however, 
provided only as examples. Each test activity has used these 
examples and their own quality assurance history and labor 
rates to calculate local TEAs. Therefore, the TEAs assessed 
in the test of RYG are tailored to the expected costs of each 
eaeervity. Since the TEAs represent a fair estimate of the 
costs each activity expects to incur because of increased 
quality assurance requirements (calculated at the time of 
contract award), this researcher believes that they would be 
considered reasonable in the event of a protest. In addition, 
Since the TEAS assessed correspond to the specific quality 
assurance actions required in the contract, they could be con- 
Sidered to be tailored to each contract. Based on the case 
law concerning liquidated damages, a tailored assessment might 
not be necessary, however, it probably strengthens’ the 
argument justifying the reasonableness of assigned TEAs. 

The second consideration in the assessment of the 
propriety of using TEAS is whether it is fair for the 
Government to consider the costs of additional quality 


assurance requirements in source’ selection. There is 
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precedent for considering costs similar to TEAs in the source 
selection decision. Specifically, the FAR discusses “ime 
consideration of first article test and transportation costs 
in the source selection process. 

Regarding first article testing, the FAR states, 
"Solicitations containing a testing and approval requirement 
shall...include, when the Government is responsible for first 
article testing, the Government's estimated testing costs as 
a factor for use in evaluating offers...." fRef. 28:para. 
9.306] In addition, the FAR requires that the solicitations 
"inform offerors that the requirement may be waived when 
supplies identical or similar to those called for have 
previously been delivered by the offeror and accepted by the 
GOVErMMent. 1 | [Ref. 28:para. 9.306] These two statements 
suggest that not only should the Government consider the costs 
of testing in the source selection decision, but that the 
decision concerning the requirement of first article testing 
should be based on the performance history of the offerors. 
Both support the use of TEAS when evaluating past performance. 

Regarding transportation costs, the FAR states, 
"Solicitations, when appropriate, shall specify that offers 
may be f.o.b. origin, f£.0.b. destination, on both; “and” vide 


they will be evaluated on the basis of the lowest overall cost 


to the Government." [{Ref. 28:para. 47.305-2] The FAR further 
States, "When evaluating offers, contracting officers shall 
consider cranspeprawaaon and transportation-related 
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costs..." [Ref. 28:para. 47.306] These statements strengthen 
the argument that it is fair for the Government to use TEAS 
based on differences in the costs it will incur because of 
quality differences. 
3. Use of Responsibility-Related Factors 

The argument against use of past performance as a 
factor in source evaluation states that since past performance 
is one of seven responsibility factors listed in the FAR, its 
use in source evaluation constitutes a responsibility 
determination [Ref. 28:para. 9.104-1]. In the case of small 
businesses, a determination of non-responsibility must be 
referred to the cognizant Small Business Administration (SBA) 
Regional Office [Ref. 28:para. 19.602-1]. Therefore, if use 
of past performance in source evaluation were considered to 
constitute responsibility determination, any negative impact 
the RYG test procedures had on small business would have to be 
referred to the SBA. Failure to comply with FAR Part 19.602- 
1 would violate statute and regulation. In order to determine 
whether the responsibility argument iS valid, several 
pertinent General Accounting Office (GAO) protest decisions 
have been examined. The cases and their implications are 
summarized below. 

Three of the cases evaluated indicate that 
responsibility factors may be used in source evaluation 


provided that negotiated procurement practices are used, and 
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the selecting activity uses these factors to make a 
comparative evaluation of offerors. 

In the matter of B & W Service Industries, Inc. (B & 
W), B & W contended that responsibility factors (i.e., company 
experience, past performance, and the experience of key 
personnel) should not have been used in the Department of 
Housing and Urban Development's evaluation of their proposal. 
GAO denied the protest. (Ref. 31] In its decision, GAO 
stated: 


Contracting officers evaluate prospective contractors to 
determine their responsibility, that is, their capability to 


perform the work....COCc (Certificate of Competency) 
referrals to SBA are only required where contracting 
officers find small businesses to be nonresponsible. In 


this case, the agency did not find the protester 
nonresponsible, but considered its proposal to be weak under 
some evaluation factors listed in the solicitation. 

With regard to these factors, it is not improper in a 
negotiated procurement to include traditional responsibility 
factors among the technical evaluation criteria....As long 
as the factors are limited to areas which, when evaluated 
comparatively, can provide an appropriate basis for a 
selection that will be in the government's best interest, 
COC procedures do not apply to a technical proposal 
deficient in those areas. [Ref. 31] 

In the case of Utah Geophysical Inc. (UGI), UGI 
contended that the Nuclear Regulatory Commission (NRC) failed 
to refer a non-responsibility determination to the SBA. They 
argued that because source evaluation included responsibility 
factors (1.e., experience and level of effort for key 


personnel assigned to the project), the NRC's finding that 


their proposal was unacceptable constituted a determination of 
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non-responsibility. pRet<. 32] GAO denied the protest and 
stated in its decision: 
Matters which are normally considered in responsibility 
determinations may properly be considered in evaluation of 
proposals when negotiation procedures are used and agency 
makes relative assessment of competing offerors' ability to 
meet contract requirements. [Ref. 32] 

In the matter of Electrospace Systems, Inc. (ESI), ESI 
protested the Army's use of offerors' experience in source 
evaluation. This evaluation criterion impacted ESI's 
exclusion from the competitive range. They argued that use of 
responsibility-related factors in source evaluation violated 
the SBA's statutory authority to certify the responsibility of 
small businesses. (Ref. 33] In its decision, GAO cites a 
previous case involving SBD Computer Services Corporation, B- 
moe), December 21, 1976, 76-2 CPD 511. In its 1976 
decision, GAO stated: 

In many other cases, we have recognized that contracting 
agencies may properly utilize evaluation factors which 
include experience and other areas that would otherwise be 
encompassed by offeror responsibility determinations when 
the needs of those agencies warrant a comparative evaluation 
of those areas. pete ceo 5) | 
In its decision to deny ESI's protest, GAO stated: 
Since neither 10 U.S.C. 2304(g) nor applicable regulation in 
any way restrict the “other factors" that may be used by 
agencies in selecting the proposal having the greatest value 
to the Government, we have not prohibited procuring agencies 
from using responsibility-related factors in making relative 
assessments of the merits of competing proposals. [Ref. 33] 
The cases cited above all support use GF 


responsibility factors in source evaluation. However, care 


must be exercised to ensure that, if such factors are used, 
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the Government makes a comparative evaluation of offerors, not 
a pass/fail determination. If a responsibility factor is used 
to ascertain the technical acceptability of a proposal, its 
use will be considered to constitute a responsibility 
determination. Two cases providing examples of this 
distinction are described below. 

In the case of Sanford and Sons Company (Sanford), 
Sanford protested the Army Corps of Engineers' determination 
that their proposal was technically unacceptable. The request 
for proposals (RFP) stated that the contract would be awarded 
to the lowest-priced, technically-qualified offeror. Two of 
the six criteria for source evaluation detailed in the RFP 
were: (1) organization and administration; and (2) 
satisfactory performance record. The Army found Sanford's 
proposal to be technically unacceptable in both organization 
and administration and past performance. Both of these areas 
are considered responsibility factors. GAO sustained the 
protest. [Ref. 34] In its decision GAO stated: 

We have cautioned that an agency may not find that a small 
business is nonresponsible under the guise of a relative 
assessment of responsibility factors and thus-~= avoid 
referring the matter to the Small Business Administration. 
[Ref. 34] 
The decision further states: 

Although an agency may use traditional responsibility 
factors, like prior performance, as technical evaluation 
factors where its needs warrant a comparative evaluation of 
proposals, an agency's rejection of a small business firm's 


offer as unacceptable under such factors was improper where 
the agency's decision did not reflect a relative assessment 
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of the offer but instead effectively constituted a finding 
of nonresponsibility. [Ref. 34] 


In the matter of Siems International Electron 
Microscope Service (Siems), Siems protested the cancellation 
of a RFP by the National Institutes of Health (NIH). NIH had 
determined that Siems was non-responsible under the RFP, but 
Siems was subsequently issued a Certificate of Competency by 
the SBA. NIH stated that the solicitation was cancelled 
because the specification in the RFP was inadequate, not 
whether or not Siems was responsible. NIH determined, 
therefore, that the requirement should be resolicited. Under 
the new RFP, Siems proposal was determined to be ‘technically 
unacceptable. This determination was based on a 
Pesponsibility factor--contractor capacity. [Ref. 35] GAO 
sustained the protest, stating: 

Where offerors were not required to submit technical 
proposals to service electron microscopes but only to offer 
to conform to the best practices of the industry, and the 
factors making up the technical criteria were evaluation of 
Capacity factors, the determination an offeror was 
technically unacceptable amounted, in essence, to a 
determination of nonresponsibility for reasons of capacity 
ema t required a referral to the Small Business 
mamainistration.... [Ref. 35] 

The conclusion that can be drawn after examination of 
the above five cases is that responsibility factors may be 
used in source evaluation, but that activities must be careful 
not to use such factors to determine the technical 


acceptability of a proposal. If implemented as written, the 


RYG test procedures do not use past performance to determine 
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technical acceptability. It will be used, however, to better 
determine the true cost to the Government of doing business 
with each offeror. In fact, the test procedures specifically 
state that a contractor's status under the test is not to be 
used aS a substitute for the requirement to make a 
responsibility determination [Ref. l:encl. (1), p. 4]. The 
test activities, however, must exercise care in administering 
the RYG procedures to ensure that they do not use a 
contractor's past performance to judge his’ technical 
acceptability. 

In summary, the Government faces a risk that GAO may 
sustain a protest arguing that use of the RYG test procedures 
constitutes a responsibility determination. However, if the 
test procedures are implemented as intended, the Office of 
General Council (OGC) in the Office of the Assistant Secretary 
of the Navy (Shipbuilding and Logistics) (ASN(S&L)) believes 
that the Government has a good argument to support its case 
that use of past performance under the RYG test constitutes a 
comparative evaluation of offerors, not a responsibility 
determination [Ref. 30}. 

4. Accuracy of the Data Base 

The validity of the Contractor Evaluation System (CES) 
data base presents the greatest potential for protest since 
they are the basis of categorizing contractors as Red, Yellow, 
or Green. The data are maintained very well by the Naval 


Material Quality Assessment Office (NMQAO). Therefore, the 


To 


CES is an accurate portrayal of the data received by NMQAO 
from the sources described in Chapter II. [Ref. 36} Problems 
will occur, however, if the data are inaccurate when they are 
Submitted to NMQAO. Culpability (Government or contractor) 
for rejects and deficiencies is assigned before the data are 
submitted to NMQAO. Because of the volume of data received, 
it would be impossible for NMQAO to verify each item [Ref. 
SiON]. If an activity submits inaccurate data, a contractor 
could be mistakenly classified Red or Yellow. If a contractor 
were to lose a contract because oe an Ineerrece: 
Classification, he would have a strong argument EO SUuppOoLeE 
filing a protest. 

Two issues arise in evaluating the probability that 
the RYG test procedures will survive a protest based on 
inaccurate data. First, the current accuracy of the data base 
must be considered. Prior to NMQAO classifying contractors 
Red, Yellow, or Green, letters were mailed to all Red and 
Yellow contractors informing them of their status and the 
reasons for the classification. With each monthly update of 
the data base, contractors are notified of any change in their 
status under the test. Contractors were advised to contact 
NMQAO if they had any questions about the program or their 
Color classification. NMQAO mailed 2037 notification letters 
and has received 247 oral and written responses to the 
classifications assigned. Thirteen of these responses 


resulted in changes to the data base. In three cases, data 
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corrections resulted in a change of color classification. 
(Ref. 37] Thirteen corrections out of 2037 would suggest a 
0.56 percent error rate. However, such an assumption would be 
invalid since only a small percentage of the contractors 
notified provided a response. 

The only way to estimate the accuracy of the data base 
is to draw a representative sample of contractors and verify 
the data leading ™= to thelr” color ™ellassitiveation- Such Var 
undertaking would require more resources than are currently 
available to NMQAO. Additionally, even if such an evaluation 
were to be made, there is no yardstick available to determine 
What would be an acceptable level of accuracy. (Ref. 36] 
Therefore, the validity and acceptable accuracy level of the 
data base can only be determined during the test. 

One of the benefits of gaining approval from the 
Defense Acquisition Regulatory Council for the one-year test 
of the RYG program is that GAO has traditionally declined to 
hear protests on other test programs [Ref. 38]. It is hoped 
that a test period free of protests will allow NMQAO to assess 
the validity of the data and resolve inaccurate data, and 
hopefully expand the program to all of DOD. 

The second issue pertaining to the accuracy of the 
data base concerns the notification letters sent to 
contractors informing them of their status in the program. 
These letters inform contractors of their current color 


classification based on documented performance history. In 
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the opinion of Ms. Sandra Desbrow-Jensen of the OGC at 
ASN(S&L), if a contractor does not respond within a reasonable 
period to correct errors in the data base, he will be estopped 
from later protesting the validity of the data base [Ref. 30]. 
Pero supports this position, the contractor notification 
letters should help eliminate protests based on the accuracy 


of the data. 


on PeOGRESS OF RYG TEST 
1. Implementation 

In order to assess the progress of the RYG program 
implementation, the researcher conducted interviews with 
personnel at each of the five test activities. The purpose of 
the interviews was to gain the perspective of both contracting 
and quality assurance personnel involved in RYG 
implementation. Therefore, interviews were conducted with 
individuals from both areas at each activity. 


The following paragraphs summarize the comments 


received during the interviews. The perspective of 
contracting personnel are presented first, followed by 
comments from the quality assurance community. Finally, the 


researcher discusses similarities and differences between 
these comments and their implications. 
eae Ont Tract ing 
(1) Naval Suppl Center (NSC) Pensacola. The 


researcher interviewed CDR Walsh, Contracting Officer at NSC 
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Pensacola when the RYG implementation began. He explained 
that the NSC had begun RYG implementation with a small number 
of Federal Supply Classes (FSCs). The Contracting Department 
chose 20 FSCs from a list given to them by the Naval Aviation 
Depot (NADEP) Pensacola. Initially, they have applied the RYG 
test procedures to ten of the 20 FSCs and small purchases. 
They intend to later expand the program to all 20 FSCs and 
major procurements. The test has not generated a great deal 
of activity to date, but CDR Walsh anticipates a greater 
impact when the program is expanded. [Ref. 39] 

The test procedures have not yet resulted in 
Significant extra work. Quality assurance personnel at the 
NADEP have helped absorb some of the extra burden. The NSC 
has not received much negative feedback from contractors. 
[Ret sso oF) 

CDR Walsh expressed concern about’ the 
accuracy of the data base used for the RYG test. 
Additionally, he worries about the affect the program will 
have on distributors who are also small businesses. Although 
theoretically distributors should be responsible for the 
quality of material they sell, small businesses may not have 
the capability to properly test their stock. He fears that 
the RYG program may hurt these small businesses. [Ref. 39] 

(2) NSC Charleston. Mrs. Davidson, Deputy 
Director, Purchase Division, Regional Contracting Department, 


NSC Charleston, was interviewed to gain the perspective of 
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contracting personnel at the other NSC involved in the RYG 
test. She explained that the RYG test procedures are being 
applied to procurements of Controlled Industrial Material 
(CIM) for the Charleston Naval Shipyard. All solicitations 
for CIM are done in writing, including small purchases, so it 
takes 30 days to receive proposals. As a result, no awards 
have yet been made using the RYG test procedures. [Ref. 40] 

Mrs. Davidson believes that evaluation of 
proposals using the test procedures will involve extra work 
for contracting personnel. However, since no evaluations have 
yet been performed, she does not know how burdensome this 
extra requirement will be. She thinks the program will be 
beneficial because it will allow the NSC to resolve quality 
problems before contract award; therefore, she expects RYG to 
reduce contract administration activities. [Ref. 40] 

Mrs. Davidson expressed concern that 
application of the RYG procedures would have a negative impact 
on the department's Productive Unit Resourcing System (PURS) 
statistics because it adds work before contract award which 
will not be given appropriate credit. Time savings are not 
realized until the contract administration phase of the 
procurement process. Therefore, although she anticipates 
benefits from the program, she wonders whether they will 
outweigh the program's negative impacts. [Ref. 40] 

(3) Ships Parts Control Center SPCC) . The 


researcher interviewed Mr. Minahan, Director of the Hull, 
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Mechanical, and Electrical Contracting Department, SPCC. He 
explained that SPCC has applied the RYG test procedures to 
five FSCs. So far, his department has not been burdened with 
much extra work. [Ref. 41] 

Mr. Minahan indicated that the procurements 
included in the RYG test at SPCC involve fairly high dollar 
values. As a result, he does not believe the application of 
TEAS will have much impact on their source selection 
decisions. He thinks the TEAs may have a greater effect at 
other activities making smaller dollar value procurements. 
PRee. aim 

(4) Naval Air and Engineering Center (NAEC). Mr. 
Fackenthal, Supervisory Contract Specialist at NAEC Lakehurst, 
was interviewed. He explained that NAEC is applying the RYG 
small purchase method as outlined in the test procedures. 
However, they requested and received approval to follow the 
Fixed Price-Greatest Value procedures instead of the 
procedures using TEAs for their major procurements. NAEC made 
this request after reviewing a sample of the contractors they 
have done business with during the last two years. Their 
review indicated that only ten percent of the contractors in 


the sample were in the CES data base. NAEC attributes this to 


the nature of their procurements. They make very few 
repetitive buys. Many of their procurements have production 
lead times of up to one year. Therefore, the performance 


history generated is too old to classify a contractor as Red, 
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Yellow, or Green. NAEC believes that the more subjective 
Fixed Price-Greatest Value procedures will better suit their 
procurements. [Ref. 42] 

Mr. Fackenthal indicated that it is difficult 
to assess the results of the program yet because they only 
began using the small purchase procedures in October 1989 and 
have not yet made a contract award using the major purchase 
procedures. However, he stated that all the comments he has 
received about the program, including those from contractors, 
have been positive. He indicated that the contracting 
personnel at NAEC are positive about the RYG program. They 
like the idea of being able to use past performance in source 
evaluation. He foresees that the program will encourage 
contractors with poor performance records to improve the 
quality of the material they deliver to the Government. ([Ref. 
42] 

(5) Naval Avionics Center (NAC). The researcher 
interviewed Mr. Wilson, Manager of Acquisition and Improvement 
at NAC. He commented that the RYG test was progressing very 
well at NAC, and that they are considering expanding the 
number of FSCs procured under the program. They are trying to 
tailor the test procedures to their operation. [Ref. 43] 

Mr. Wilson indicated that the RYG test has 
not resulted in a great deal of extra work for contracting 
personnel at NAC. He believes that the ease with which they 


implemented the program is due in part to the culture at NAC 
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which welcomes innovation. He would like to see the CES 
expanded to the Buyer Information System described in Chapter 
II. He hopes that the validity of the data base can be proven 
during the test so that in the future it can be used by 
COnerEacking personnel Leo. perform responsibility 
determinations. [Ref. 43] 

Mr. Wilson explained that most of NAC's 
contractor quality problems have been eliminated by 
implementation of the Blue Ribbon Contractor program. 
Therefore, they have not been able to determine whether the 
RYG test program has caused contractors to improve the quality 
of the material they supply to NAC. He also indicated that 
field activities can make errors impacting on a contractor's 
color classification: In one instance at NAC, some test 
equipment was improperly programmed. The error resulted in 
the erroneous rejection of material supplied by a vendor on 
NAC's Blue Ribbon Contractor list. The rejects at NAC caused 
the contractor to be classified Red in the RYG data base. The 
contractor did not lose any contracts because of his incorrect 
color classification. However, it took NAC a couple of months 


to investigate and correct the problem after the contractor 


questioned his color classification. [Ref. 43] 
b. Quality Assurance 
(1) NADEP Pensacola. Mr. Hargett, Quality 
Developmental Project manager at NADEP Pensacola, was 


interviewed to gain perspective on the progress of the RYG 
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test implementation in Pensacola. Mr. Hargett explained that 
Pensacola had purposely limited the initial application of the 
RYG test procedures to allow personnel involved to become 
accustomed to them. Their slow start has resulted in little 
activity from the test to date. However, involvement in the 
test program has caused them to increase the amount of quality 
data submitted to NMQAO. As a result, they have added 
approximately 1000 vendors to the CES data base. He expects 
this increase to cause more activity under the test in the 
future. [Ref. 44] 

Mr. Hargett indicated that the RYG test 
procedures increase the workload of the quality assurance 
personnel, but they do not mind devoting the required time. 
Because of the test, Mr. Hargett has become deeply involved 
with the contracting personnel at NSC Pensacola. He said this 
is the first time the two organizations have worked together. 
Mr. Hargett has been involved with the RYG program since its 
conception. Although he was positive about the test at the 
outset, he stated that he iS even more positive about the 
program now. (Ref. 44] 

(2) Charleston Naval Shipyard. The researcher 
interviewed Ms. Cassell, Supervisory Quality Assurance 
Engineer at the Charleston Naval Shipyard. She indicated that 
the test is beginning slowly there. She believes that most of 
the delay has been caused by unrealistically low cost 


estimates placed on purchase requests by the Shipyard's 
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material planners. She explained that often the planners 
estimate costs to be less than $2500. As a result, the 
purchase requests are not included in the RYG test. When 
material is procured, however, the actual costs are usually 
greater» than S2500: Therefore, procurements that should 
qualify for the RYG test are not included because of 
inaccurate cost estimates. She indicated that the Quality 
Assurance Department at the Shipyard has been working with the 
Planning Department to correct this problem. [Ref. 45] 

Ms. Cassell iS positive about the RYG 
program. She believes it highlights the Shipyard's quality 
assurance problems to the contracting personnel at NSC 
Charleston. She also thinks, however, that the location of 
contracting and quality assurance in different commands causes 
the Contracting Department at the NSC to be out of touch with 
the quality problems experienced by the Shipyard. She 
believes this separation may hinder the implementation of the 
RYG test. [Ref. 45] 

(3) Spcc. The researcher interviewed Mr. Jornovy, 
Quality Assurance Specialist, Level 1/SS Section, SPCC. He 
indicated that SPCC has used the RYG procedures in 
approximately 30 solicitations, but has not yet received any 
material under the program. So far, they have not received 
any complaints from contractors. He explained that they have 


limited the implementation to cover only spare parts. They 
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later expect to expand the program to include valves, their 
Major commodity. [Ref. 46] 

Mr. Jornov said that the increased work load 
has not been burdensome for them, involving approximately one 
hour each day. However, the work load will increase if the 
program is expanded. He thinks the RYG program is worthwhile. 
He believes it needs some fine tuning, but that eventually it 
will encourage vendors to improve the quality of their 
material. He indicated that he has misgivings about the data 
base. He is concerned that some activities may not send data 
to NMQAO and that NMQAO may not enter all the data received. 
He fears that an incomplete data base may distort contractor 
@eror Classifications. [Ref. 46] 

(4) NAEC. Mr. Robert Armitage, Quality Assurance 
Specialist, Procurement pivis Ton, Quality Assurance 
Department, NAEC, was interviewed. He indicated that the 
extra work required by the RYG test has not been too 
burdensome for quality assurance personnel at NAEC. He is 
very positive about the program. He believes it is a useful, 
Viable tool for NAEC. He would like to see the impact the 
program has on contractors with poor quality history 
strengthened. However, he is glad that the program allows 
NAEC to consider gquality history to a greater degree than they 
had in the past. He said that any new tool that helps them do 


their job better is welcome. [Ref. 47] 
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(5) NAC. The researcher interviewed Ms. Powell, 
Electronics Engineer, Quality Assurance Department, NAC. She 
indicated that the bulk of the extra work created by the RYG 
test has been related to handling contractor complaints. 
Investigating complaints generally takes between one and two 
weeks. Ms. Powell believes the program is an excellent 
concept. It forces commands to correct their data. im 
addition, before the RYG test, contractors did not know (or 
care) whether their quality history was good. Now they do. 
Implementation forces Government activities to communicate 
with contractors. [Ref. 48] 

Ms. Powell indicated that some of the 
contractor complaints have proven to be valid. Therefore, she 
believes the weakest part of the program is the data base. 
Specifically, she thinks problems may occur in placing 


liability (Government or contractor) for quality deficiencies. 


[Ref. 48] 
c. Analysis of Interviews 
None of the people interviewed exhibited a 
negative attitude concerning the RYG test program. Jig 


general, quality assurance personnel appeared to be more 
positive about the program than contracting personnel. 
Although the test requires extra work of both communities, 
quality assurance personnel seemed more convinced that the 
benefits derived from the program would outweigh the extra 


effort. Perhaps their perspective results from a feeling that 
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for the first time they have a tool to influence the quality 
of material before it is received. The amount of inspection 
and documentation quality assurance performs varies inversely 
with the quality of material received. Therefore, they 
probably perceive that improved quality will simplify receipt 
inspection and reduce documentation of quality deficiencies. 
This will make their job easier. 

Two of the people involved in contracting, 
however, were aS positive about the program as were quality 
assurance personnel. These two people work at the two Navy 
Industrial Fund activities involved in the test, NAC 
Indianapolis and NAEC Lakehurst. The two contracting people 
from the NSCs at Pensacola and Charleston were the two most 
concerned about the program. One of the reasons for the 
difference in opinion on this program was based on the PURS 
used at NSCs. As explained by Mrs. Davidson of NSC 
Charleston, this system rewards the NSC for the speed with 
which they award contracts and purchase orders. The RYG test 
procedures slow down the award process, promising improved 
quality at the end of the procurement process. Therefore, 
implementation of the RYG program may penalize activities 
subject to PURS guidelines. A second reason for the 
difference in attitudes may be the location of the contracting 
and quality assurance personnel at each test site. 
Contracting personnel at NSC Pensacola must work with quality 


assurance personnel at NADEP Pensacola; contracting personnel 
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at NSC Charleston must work with quality assurance personnel 
at the Charleston Naval Shipyard. The NSCs and industrial 
activities they support have different missions and different 
commanding officers. These distinctions probably make it more 
difficult for the two functional areas to undeérstanden. 
other's concerns and to make the RYG program succeed. 

Aside from helping the activities receive better 
quality material, the interviewees foresee two peripheral 
consequences of the RYG program that are positive. First 
they expect the program to encourage Red and Yellow 
contractors to improve the quality of material they supply to 
the Government. Second, they indicated that implementation of 
the test procedures has forced contracting and quality 
assurance personnel to work together. The increased contact 
1S expected to help the two groups better understand each 
other's problems. The end result should be that both groups 
are able to do their jobs more effectively. 

The biggest area of concern voiced by the test 
activities is the validity of the data base. This worry was 
expressed by NSC Pensacola, both quality assurance and 
contracting at NAC Indianapolis, and quality assurance at 
SPcc. This concern corresponds to the assessment of personnel 
at the Naval Supply Systems Command and OGC at ASN(S&L) 
(described earlier in this chapter) that the greatest 
potential for protest of the program lies with the accuracy of 


the data base. 
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2. Test Results 

The one-year test of the RYG program began 1 August 
1989. Not all of the five activities began using the RYG test 
procedures on this date, however. Moreover, when they began 
the test, several of the activities limited the initial number 
of procurements included in the test. Limited implementation 
allows the activities to fine tune their internal procedures 
before attempting full-scale implementation. Although this 
approach is based on sound judgment, it has the peripheral 
effect of limiting the amount of available data documenting 
the results of the RYG test. 

The researcher has collected the data that are 
currently available; they are summarized below. Although the 
data are scant, an attempt has been made to analyze them and 
assess their implications on the progress of the RYG test. 
The data pertaining to the number of Red (R), Yellow (Y), 
Green (G), and Insufficient Data (I) contractors will be 
presented first. Next, data concerning the status of 
procurements under the program will be reported. 

Table 1 summarizes the numbers of letters sent to 
contractors advising them ane their initial EColior 
classification and subsequent status changes and the responses 
received by NMQAO. Tihesdatamina table  lWwaindicate that the 
initial color classification resulted in 1033 Red and Yellow 


contractors. Subsequent monthly updates to the RYG data base 
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TABLE 1 


RED YELLOW GREEN NOTIFICATION AND RESPONSE 





Month Notification Sent Response Received 
May 1033 179 
July 655 23 
August 348 35 
September FATES me, 
Lowa» 2507 247 


Source: LRefi. 373 pauee 


changed contractor color Vellassilitlcauens. Each update has 
resulted in fewer contractors changing status. 

Table 2 describes the categories into which the 
responses fell and the action taken by NMQAO in reaction to 
them. The data in Table 2 indicate that contractors disagreed 
with the data or their color classification in 49 cases (40 
disputed the data and nine disagreed with their color 
classification). NMQAO determined that their disagreements 
were valid and changed the data in 13 cases. Three of the 13 
Gata base corrections resulted in a change in color 
classification. Although 2307 contractors were notified, only 
247 (or 10.7 percent) responded. Therefore, the errors 
discovered because of these responses cannot be used to 


estimate the accuracy of the data base. 
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TABLE 2 


RESPONSES~--CATEGORIES AND ACTION TAKEN 


Category of Response Quantity 
Information a 
Unaware of Rejects 2 
Disputed Data 40 
Classification Disagreement 9 
Action Taken Quantity 
No Action 83 
Forwarded Data Profiles 108 
Referred to Other Command 48 
Conducted/Scheduled Meeting 5 
Changed Data ES 
Changed Classification Ss) 


Source: GREE +3377. | 


The first group of procurements made under the RYG 
test are summarized in NMQAO's September 1989 status report. 
The data relating to these procurements are detailed in Table 
te The data in Table 3 indicate that in most cases the 
lowest-priced proposals were submitted by Green and 
Insufficient Data contractors. Of the 77 procurements made 
under the RYG test, Red and Yellow contractors submitted the 
lowest-priced proposals in only ten instances (seven displace- 


ments of Red and Yellow contractors plus three awards to 
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TABLE 3 


RED/YELLOW/GREEN PROCUREMENTS 


Awards to RYGI Contractor 
Contractors Displacements 
Noten iney. R ue G ae R Y 
NAEC Lakehurst 0 0 0 0 0 0 
NAC Indianapolis 0 Meg Ae 0 5 
SPCC Mechanicsburg 0 O 18 0 Z 0 
NSC Charleston 0 0 0 0 e) 0 


NSC Pensacola 


lo 
lo 
B 
=) 
he) 
lo 
lo 


Total 0 30 456829 2 5 


Source: [Ret. 379: pp. =o) 


Yellow contractors). These ten cases represent 13 percent of 
the 77 procurements. The awards made to Yellow contractors 
indicate that the Government's consideration of the costs of 
quality does not preclude contractors with poor quality 
histories from receiving Government contracts. In other 
words, a contractor's color classification does not eliminate 
him from the competitive range. 

It iS too early to determine whether the RYG program 
will improve either the quality of material the Government 
receives or contractors' motivation to deliver high quality 
material. It is also impossible to judge the reaction 
contractors will have to displacements caused by the program. 


The data do show, however, that the program is having some 
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affect on contract awards; Red and Yellow contractors have 
lost contracts to higher priced contractors with better 
performance histories. The data also show that the test 
activities are not using the RYG data base to eliminate 
contractors from the competitive range. This should support 
the Government's position that application of the RYG formula 
in source evaluations does not constitute a responsibility 


determination. 


18: SUMMARY 

This chapter assessed the strengths and weaknesses of the 
RYG program. To accomplish this, it first discussed the 
ability of the RYG program to withstand protests based on four 
arguments--de facto debarment, TEAS as penalties, 
responsibility determination, and validity of the data base. 
Second, it discussed the progress of RYG test implementation 
at the five test activities. Interviews were conducted with 
personnel involved with the test at each of the activities and 
data related to the test were analyzed. Interviewees 
generally felt that the RYG program would yield positive 
results. However, several of them expressed concern about the 
mamerdity of the data in the CES. The final chapter will 


present the conclusions and recommendations of the thesis. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. INTRODUCTION 

This chapter will discuss the conclusions and recommenda- 
tions resulting from the thesis research. After presenting 
the conclusions and recommendations, the research questions 
will be answered. Finally, recommendations will be made 


concerning areas for further research. 


B. CONCLUSIONS 


1. The Red Yellow Green (RYG) Program is Appropriate 
for Use in Field Contract 1 ngronim 


The RYG test procedures incorporate a formula driven 
assessment or CONt bacreor past performance in source 
evaluation. The formula used in the test was designed to 
assess contractor quality history for groups of relatively 
simple commodities. The test procedures were intended to be 
easy to apply and, therefore, involve very little subjective 
input from contracting and quality-assurance personnel. Major 
system contracting would require a much more subjective 
assessment of contractor-quality history than allowed by the 
RYG program, 


2. The Air Force Contractor Performance Assessment 


Reporting System (CPARS) is Appropriate for Major 
Systems Procurement Only 


CPARS should not be expanded for use in field 


contracting. Maintenance of the data for CPARS is very labor 
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intensive. The personnel resources required to maintain an 
expanded CPARS would prohibit including enough contractors in 
the system to make it useful for field contracting. In 
addition, use of CPARS in source evaluation requires time and 


personnel not available in field contracting offices. 


Ss. Blue Ribbon Contractor Programs are Appropriate for 
Use at Field Activities in Which Contracting and 
Quality Assurance Personnel are Able to Work Together 


Unlike RYG, the assessment of contractor performance 
history under a Blue Ribbon program is not formula-driven. 
The decision to award to a Blue Ribbon contractor over a 
lower-priced offeror (not included in the program) requires 
an assessment by both contracting and quality-assurance 
personnel. Therefore, the two groups must communicate with 
each other to use the program effectively. 


4. RYG and Blue Ribbon Contractor Programs Can Be 
Implemented Together and Can Enhance Each Other 


A Blue Ribbon program would be used when procuring 
critical goods and services because it allows an activity to 
concentrate its attention on those parameters that it 
considers important. RYG would be used for all other 
procurements which do not justify devotion of the resources 


needed to apply a Blue Ribbon program. 


5. The RYG Program (Especially if Implemented in Conjunc- 
tion with a Blue Ribbon Contractor Program) 1S a 
Realistic Method for Initiating Total Quality 
Management (TOM) in Field Contracting 


The time and personnel required to perform vendor TQM- 


assessments make complete TQM implementation in field 
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Contracting ilpsacracam However, the RYG and Blue Ribbon 
Contractor programs allow field Contracting activities gee 
consider contractor past performance in the source selection 
decision. Therefore, they allow activities to comply, at 
least partially, with one of Deming's 14 principles: "End the 
practice of awarding business on the basis of price tag. 
Instead, minimize total cost. Move toward a single supplier 
for any one item, on a long-term relationship of loyalty and 


Crus tie es Refi 0. pi 2 ca 


6. Application of the RYG Test Procedures Does Not 
Constitute De Facto Debarment 


Use of the test procedures neither stops’ the 
Government from soliciting offers from nor awarding contracts 
to Red and Yellow contractors. If, after considering the 
costs related to additional quality assurance requirements, a 
Red or Yellow contractor is the lowest priced responsible 
offeror, the Red or Yellow contractor will be awarded the 


GCOneEraACE: 


re The Assessment of Technical Evaluation Adjustments 
(TEAS) Under the RYG Test is Reasonable and Does Not 
Represent a Penalty to Red and Yellow Contractors 
There is precedent for considering costs similar to 
TEAS in source evaluation (i.e., consideration of first- 
article test and transportation costs). Therefore, it is 
reasonable for the Government to consider the costs of 


additional quality assurance requirements in source selection. 


TEAs are calculated using historical data for labor hours 
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required for quality assurance actions, labor rates, and costs 
of testing. Since their calculation is based on actual costs 
the Government would expect to incur when forced to carry out 
additional quality assurance actions, TEAs should not be 


considered penalties. 


8. If the RYG Test Procedures are Implemented as 
Intended, the Government Should Be Able to Support 
the Argument that the Use of Past Performance Under 


the Test Does Not Constitute a Responsibility 
Determination 


Precedent from cases in which contractors protested 
the use of responsibility-related factors in source evaluation 
indicate that responsibility factors may be used in source 
evaluation. However, activities must be careful to use such 
factors to perform a comparative evaluation of offerors, not 
to determine the technical acceptability of proposals. The 
RYG test procedures use past performance to better determine 
the true cost to the Government of doing business with each 
offeror, not to determine the technical acceptability of 


proposals. 


weeene Validityoot the Contractor Evaluation System (CES) 
Data Base Presents the Greatest Potential for Protest 
of the RYG Test Procedures 


Personnel at the Naval Supply Systems Command and at 
several of the activities implementing the RYG test expressed 
concern regarding the accuracy of the data in the CES. 
Specifically, they fear that if an activity were to submit 
inaccurate data, a contractor could be mistakenly classified 


Red or Yellow. MecerCOncractemeevweressco lose a contract 
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because of an incorrect classification, he would have a strong 
argument to support filing a protest. 


10. It is Possible that Red and Yellow Contractors Wiese 
Estopped from Protesting the Validity of the CES 


Each Red and Yellow contractor is notified of his 
color classification and the performance history behind the 
designation. He is given an opportunity to challenge the data 
supporting his status under the RYG program. If a contractor 
is aware that the data justifying his classification are 
incorrect, and he does not respond to correct them within a 
reasonable period, he may give up the right to challenge these 


errors later during a protest. 


C. RECOMMENDATIONS 
1. Evaluation of RYG at the End of the Test Period 

The benefits of the RYG program and its ability to 
withstand a protest should be evaluated at the end of the 
program test, currently scheduled for 1 August 1990. There 
are not enough data currently available to evaluate whether 
the benefits of the RYG program outweigh the costs of 
implementation. The positive comments expressed by personnel 
at the five activities involved in the test suggest that the 
program will be beneficial. However, an assessment concerning 
whether the program improves contractor performance must 
eventually be made. Such an evaluation should include 
reviewing quality records at the five test activities to 


determine whether the quality of incoming material has 
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improved. In addition, a sample of contractors should be 
evaluated to determine whether the RYG program has encouraged 


them to improve their performance. 


2. Continuation of the RYG Program After Completion of 
the Test 


If there are not enough data to adequately assess the 
value of the RYG program at the end of the test period, use of 
the test procedures should be continued until the program can 
be properly evaluated. Contractor evaluation, as described in 
the previous paragraph, will require the passage of enough 
time for contractors to become aware of the effects of the RYG 
program, change their performance, and deliver enough material 
to the Government to document that change in performance. The 
criteria used to designate contractors as Red, Yellow, and 
Green span periods up to two years. Therefore, it is probable 
that an accurate evaluation of the affect the RYG program has 
on contractors will require more than the one year of data 
that will be accumulated under the test. Since the program 
has the potential to yield such positive results, its use 
should be continued until the program can be evaluated 
properly. 


3. Implementation of Blue Ribbon Contractor Programs with 
RYG 


If the RYG program is determined to be beneficial, 
activities should be encouraged to implement a tailored Blue 
Pebon Contractor program in conjunction with RYG. As 


discussed in the Conclusions section of this chapter, RYG and 
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Blue Ribbon Contractor programs can enhance each other. In 
addition, they both move field contracting offices toward 
implementation of TQM. Therefore, implementing them together 


should prove beneficial to the Department of Defense (DOD). 


D. ANSWERS TO THE RESEARCH QUESTIONS 


1. How is the Navy's Red Yellow Green Program Structured 
and How is it Intended to Improve the Quality of 
Material Procured by the Navy? 


The RYG program uses contractor performance history in 
source evaluation. The program uses this history in one of 
two ways. The first approach uses a contractor's quality 
history to determine whether the Government will require 
additional quality assurance actions when doing business with 
the Contractor, If the contractor's performance history is 
good, additional quality assurance actions will not be 
required; if his history is poor, additional actions will be 
required. If additional quality assurance is required, the 
costs of performance to the Government are added to the 
contractor's proposal using TEAs. The source selection 
decision is made using the proposed prices of Green 
contractors and the adjusted prices of Red and Yellow 
COMEEACEGES. 

The second approach involves a more subjective use of 
contractor performance history. A source selection plan is 
prepared by quality assurance personnel, specifying the 


criteria to be used in evaluating Red, Yellow, and Green 
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Sentractors. When proposals are received, quality assurance 
personnel assign ratings to each offeror based on the criteria 
outlined in the source selection plan. Contracting personnel 
award the contract based on the scores received by 
contractors, considering their quality and price ratings. 

The RYG program is intended to improve the quality of 
material procured by the Navy in two ways. Pairs, 
consideration of contractor quality history in = source 
evaluation should result in a higher percentage of contract 
awards to contractors with good performance history. This is 
expected to improve the quality of material received. Second, 
the program should encourage contractors with poor quality 
histories to improve their performance in order to receive 
more Government contracts in the future. 


2. How are Red, Yellow, and Green Contractors Defined? 





A Red contractor is considered a high quality risk. 
His performance history 1S poor. It warrants requirement of 
Special quality assurance actions and higher-level review 
before contract award. A Yellow contractor 1S considered a 
moderate quality risk. His performance history 1s poor enough 
to warrant requirement of special quality assurance actions. 
A Green contractor is considered a low quality risk. His 
performance history is good and, therefore, warrants no 


special quality assurance actions. 
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3. What are the Other Current Contractor Quality 
Initiatives Wi thine DoD: 


The other current quality initiatives within DOD are 
the Air Force CPARS, the Naval Avionics Center (NAC) 
Indianapolis Blue Ribbon Contractor program, and the DOD TQM 
initiative. 


A How Does RYG Compare and Contrast to These Other 
Programs? 


RYG and CPARS were designed to be used in different 
areas of procurement. RYG is intended to be employed by field 
contracting activities. CPARS is intended for major systems' 
procurement. RYG and NAC's Blue Ribbon Contractor program are 
both intended for use in field contracting. The Blue Ribbon 
program requires an activity to devote more resources to and 
employ more judgment in contractor performance evaluation than 
does RYG. It would be impractical for some activities to 
implement a Blue Ribbon Contractor program. However, 
implementation of such a program can enhance the results 
expected of the RYG program. The RYG program is consistent 
with the goals of the DOD TQM initiative. It helps field 


contracting offices to begin implementation of TQM. 


5. What are the Strengths and Weaknesses Associated with 
RvGe 


The RYG program has many strengths. It allows use of 
contractor performance history in the source selection 
decision and so should improve the quality of material 


received by the Government. It requires few resources to 
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implement and, therefore, can be used by field contracting 
offices. It forces contracting and gquality-assurance 
personnel to work together. It forces Government activities 
to communicate with industry. The program does have some 
weaknesses, however. The data base used in the program is 
very large, and its accuracy cannot be currently assessed. If 
the data base proves inaccurate, the program is built on a 
weak foundation. The program's evaluation of contractor 
performance history is formula-driven. Therefore, it is only 
appropriate for procurement of relatively simple commodities. 
6. How is the RYG Program Being Tested in the Market? 

The RYG program is being tested at five Navy field 
activities. Interviews with personnel at the test activities 
indicate that the test began slowly but is progressing well. 
Both contracting and quality assurance personnel expressed 
positive opinions about the program. There are currently too 


few data, however, to assess the success of the test. 


PeeeeeaREAS FOR FURTHER RESEARCH 

One area for further research is to evaluate the benefits 
of the RYG program after the one-year test period. More data 
will be available at this point and, therefore, a researcher 
will be better able to assess the merits of the program. 
Another area for further research is to determine how NAC's 
Blue Ribbon Contractor program could be exported to other 


activities. The Blue Ribbon program has proven very 


O° 


beneficial for NAC and could be expected to benefit other 


field contracting activities. 
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COLOR CODE 


RED A 


B 


APPENDIX A 


RED YELLOW GREEN CLASSIFICATION CRITERIA 


CLASSIFICATION CRITERIA 
ON CURRENT NAVY VDAR 
MitieD SC Dp eAND/OR EGGURRENTLY IN EFFECT 


QUALITY INFO ON LATEST PRE-AWARD SURVEY 
(PAS) WITHIN LAST YEAR--NO AWARD 


foe RODUCl—-GRIENTED SURVEY (POS) IN 
LAST TWO YEARS UNACCEPTABLE 


LATEST QUALITY SYSTEM REVIEW (QSR) IN LAST 
TWO YEARS UNACCEPTABLE 


EATES TS oPECIAL SURVEY IN LAST TWO YEARS 
UNACCEPTABLE 


PEE Cll Abels 4eO0R MORE IN LAST TWO YEARS 
BOR VLVE CRORE eels 


TWO OR MORE UNSAT FIRST ARTICLE TESTS 
(FAT) IN LAST YEAR 


TWO OR MORE CATEGORY “I" ODRS IN LAST YEAR 


SLX CR MORE CATEGORY “TI ACTION ODRS IN 
LAST YEAR 


Six OR MORE CONTRACTOR LIABLE RODS IN LAST 
VEAR 


ot On MORES CONTRACTOR LIABLE WAIVERS / 
DEVIATIONS IN LAST YEAR 


ON DLA CONTRACTOR ALEK Gist FOR MAJOR 
I eee Ire NC TES 


Ww Classification criteria L, M, N for "Red" and "Yellow" 
have not yet been incorporated. 


LOE 


COLOR 


YELLOW 


CLASSIEPICATIONS CRE rae 
ISSUED VDAR LETTER CE CONecErt 
PREVIOUSLY CLASSI EPIER RED! 


LATEST PAS IN LAST TWO YEARS--AWARD WITH 
FINDINGS 


LATEST POS IN LAST TWO YEARS ACCEPTABLE 
WITH CORRECTIONS 


LATEST QSR IN LAST TWO YEARS ACCEPTABLE 
WITH CORRECTIONS 


LATEST SPECIAL SURVEY IN LAST TWO YEARS 
ACCEPTABLE WITH CORRECTIONS 


REJECT RATE 6-143 FCR FIVE OR@MORE 
LOTS/ANY REJECT RATE FOR LESS THANE eG 
LOTS IN LAST TWO YEARS 

ONE UNSAT FAT IN LAST YEAR 

ONE CATEGORY “IUSODR ye tAs eyoek 


THREE-FIVE CATEGORY "Ii WexAcTION ODRS a 
LAST YEAR 


THREE-FIVE CONTRACTOR LIABLE RODS IN LAST 
YEAR 


THREE~FIVE CONTRACTOR LIABLE WAIVERS / 
DEVIATIONS IN LAST YEAR 


ON DLA CONTRACTOR ALERT LIST FOR MINOR 
DEFICIENCIES 


“Classification criteria L, M, N for “Red and “Yello... 
have not yet been incorporated. 
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COLOR CODE Clase CARTON, CRITERIA 





GREEN Cc LATEST PAS IN LAST TWO YEARS--AWARD WITH 

NO FINDINGS 

D LATEST POS IN LAST TWO YEARS ACCEPTABLE 

E Ear eobeebhe IN TASst TWO YEARS ACCEPTABLE 

E EXE ote or Eclat SURVEY IN LAST TWO YEARS 
ACCEPTABLE 

G REJECT RATE LESS THAN 6% IN LAST TWO YEARS 
MOnee tVETOR MORE LOTS 

H ALL FAT IN LAST YEAR SATISFACTORY 

K ONE-TWO CATEGORY "II" ACTION QDRS IN LAST 
YEAR 

L ONE-TWO CONTRACTOR LIABLE RODS IN LAST 
YEAR 

M ONE-TWO CONTRACTOR LIABLE WAIVERS/ 


DEVIATIONS IN LAST YEAR 


“Classification criteria L and M for "Green" have not yet 
been incorporated. 
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APPENDIX B 


CLAUSES FOR SIMPLIFIED SMALL PURCHASE PROCEDURES 


NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 


(a) This procurement 1s subject to a test of the Navy's 
Contractor Evaluation System (CES), "Red/Yellow/Green" (R/Y/G) 
program. The test is authorized by the Assistant Secretary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 


(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractors' past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. The 
price to be considered in determining best value will be the 
evaluated price after Technical Evaluation Adjustments (TEAs) 
for related quality assurance actions, as applicable, are 
applied to the offered price. 


(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTORS--TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 


(d) The commodities included in this test, as currently 
solicited, are: 


FSC No. FSC Nomenclature CLE 
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ADDITIONAL EVALUATION FACTORS--TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (SIMPLIFIED SMALL PURCHASE PROCEDURES) 


(a) This procurement is part of a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics), for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program's effectiveness and impact on 
the acquisition process. 


(b) The purpose of R/Y/G is to assist contracting personnel 
during source selection to determine the best value for the 
Government--price, past quality performance, and other factors 
considered. The test program uses accumulated contractor 
quality performance data to classify contractor's performance 
on select commodities as either "Red" (high risk), "Yellow" 
(moderate risk), "Green" (low risk), or "Insufficient Data," 
based on the degree of risk to the Government of receiving 
poor quality products. Such classifications are then used to 
apply a Technical Evaluation Adjustment (TEA) during source 
selection. 


(c) A TEA 1S a monetary assessment added to the price of 
selected commodities that have been classified as either "Red" 
or "Yellow" for specific contractors, and is based on the cost 
to the Government Ot effecting additional quality 
considerations that would otherwise not be required if award 
were made to a contractor with a satisfactory performance 
history. For purposes of requirements uSing the simplified 
small purchasing procedures, standardized TEAS have been 
established for the "Red" and the "Yellow" classifications. 
During evaluation of quotations, the applicable TEA is added 
to the quoted price of the "Red" and/or "Yellow" commodity, 
and after consideration of any other pertinent price-related 
factors (e.g., transportation charges, First Article Testing, 
discount terms, etc.), becomes the basis for determining award 
of the purchase order. A commodity's classification may 
change over time aS new or revised quality performance data 
become available. 


(ad) Classifications for the test program are summarized as 
follows: 


"Green"--Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 


"Vellow"--Moderate risk. History of quality problems; special 


quality requirements/actions needed; Technical Evaluation 
Adjustment (TEA) applied to offered price. 


ZO 


"Red"--High risk. Special alert to history of seer quali) 
performance; TEA applied to offered price(s), and contract 
award requires higher level approval. 


"Insufficient Data"--Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, a TEA is not assessed. 


(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Naval Material Quality 
Assessment Office (NMQAO), Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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APPENDIX C 


CLAUSES FOR MAJOR PURCHASE PROCEDURES 


Peer TO PROSPECTIVE OFFERORS (NOV 1988) 


(a) This procurement is subject to a test of the Navy's 
Contractor Evaluation System (CES), "Red/Yellow/Green" (R/Y/G) 
program. The test is authorized by the Assistant Secretary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 


(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractors' past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. The 
price to be considered in determining best value will be the 
evaluated price after Technical Evaluation Adjustments (TEAs) 
for related quality-assurance actions, aS applicable, are 
applied to the offered price. 


(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTORS--TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 


(ad) The commodities included in this test, as currently 
solicited, are: 


boc No. FSC Nomenclature GLEN 
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ADDITIONAL EVALUATION FACTORS--TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (MAJOR PURCHASE PROCEDURES) 


(a) This procurement is part of a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics), for the acquisition cf Speciin. 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program's effectiveness and impact on 
the acquisition process. 


(b) The purpose of R/Y/G is to assist contracting personnel 
during source selection to determine the best value for the 
Government--price, past quality performance, and other factors 
considered. The test program uses accumulated contractor 
quality performance data to classify contractor's performance 
on select commodities as either "Red" (high risk), "Yellow" 
(moderate risk), "Green" (low risk), or "Insufficient Data," 
based on the degree of risk to the Government of receiving 
poor quality products. Such classifications are then used to 
apply a Technical Evaluation Adjustment (TEA) during source 
selection. 


(c) A TEA 1S a monetary assessment added to the price of 
selected commodities that have been classified as either "Red" 
or "Yellow" for specific contractors, and is based on the cost 
to the Government for effecting additional quality 
considerations that would otherwise not be required if award 
were made to a contractor with a satisfactory performance 
history. During evaluation of quotations, the necessity for 
any additional quality assurance requirements will be 
determined, and the applicable TEA will be assessed onto the 
quoted price of the "Red" and/or "Yellow" commodity. After 
consideration of any other pertinent price-related factors 
(€.9., transportarion) charges tiiqse Article Testing, discount 
terms, etc.), this adjusted price becomes the basis for 
determining award of the purchase order. A commodity's 
classification may change over time as new or revised quality 
performance data become available. 


(ad) Classifications for the test program are summarized as 
follows: 
"Green"--Low risk. No extraordinary quality requirements or 


additional actions required; satisfactory quality history. 
"Yellow"--Moderate risk. History of quality problems; special 


quality requirements/actions needed; Technical Evaluation 
Adjustment (TEA) applied to offered price. 
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Brea --High risk. Special alert to history of poor quality 
performance; TEA applied to offered price(s), and contract 
award requires higher level approval. 


"Insufficient Data"--Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, a TEA is not assessed. 


(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Navy Material Quality 
Assessment Office (NMQAO), Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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CODE 


APPENDIX D 


GUIDELINE FOR TEA ASSIGNMENT 


RED CLASSIFICATION 


CLASSIFICATIONS CR IT TERUA 


On Current Navy VDAR 
(Time Frame--Six Months) 


DLA Corrective Action 

Methods CC, D, or EsCurrvenek, 
in Effect 

(Time Frame--Within Two Months) 


Latest Pre-Award Survey 
(Negative Quality) 
(Time Frame--Two Years) 


Latest Product Oriented Survey 
Rated Unsat——-Ceould impact 
Product Quality 

(Time Frame-Two Years) 


Latest Quality System Review 
Rated Unsat--Could Impact 
Product Quality 

(Time Frame--Two Years) 


Latest Quality Audit/Special 
survey 

Rated Unsat——-Could inact 
Product Quality 

(Time Frame--Two Years) 


ee 


ADDITIONAL 
OA REO'TS 


1A or 1B 
2A or 2B 
3 
4 
5 Or 26 
7 


1A or 1B 

2A or 2B 
4 

5 @ia ao 
fi 


1A or 1B 
4 

DOL es 
y 


1A or 1B 
A 

Sere 15 
7 


1A .CE Ls 

2A or 2B 
4 

BOL 6 
o 


1A or 1B 
4 

Ores 
7 





She oolPrcarTion_ CRITERIA 


Material Inspection Records 
Reject Rate 15% or Greater 
Indicated 

(Minimum Five Lots in Two Years) 


First Article Test 
Two or More Unsat 
(Time Frame--One Year) 


Quality Deficiency Reports--Cat I 
Two or More Unsat 
(Time Frame--One Year) 


Quality Deficiency Reports-- 
Caen 

Six Or Mores“AcE1on "2Gat IT 
(Time Frame--One Year) 


Report of Discrepancy 

Six or More Contractor Liable 
RODS 

(Time Frame--One Year) 


Walivers/Deviations 

Six or More Contractor Liable 
Requests 

(Time Frame--One Year) 


DLA Contractor Alert List 
Listed for Major Deficiencies 
Which -Coulde inpace seroduce 
Quality 

(Time Frame--Six Months) 


ADs 


ADDITIONAL 
OA REO'TS 


EA or 1B 


oe ey ae Bs 
ZAMOr 25 


LPancor 15 
2m Or 2B 


1A or 1B 


2hNoOr 26 


2Po Or 25 


He Or iB 
BA Or 2b 


SOY 46 





YELLOW CLASSIFICATION 


ADDITIONAL 
CLASSIFICATIONS CRITTER TA OA REO'TS 
VDAR Discussion Contractor or 1A or 1B 
Letter of Concern Issued 3 
(Time Frame--Six Months) 4 
5 Oilnee 

7 
Previously Classified "Red" I Wefone 11k ls) 
(Time Frame--Six Months) 3 

4 

5 or 6 

7g 
Latest Pre-Award Survey 1A or 1B 
Positive for Quality but 4 
Negative in 5 Gro 
One or More Other Categories 
(Time Frame--Two Years) 
Latest Product Oriented Survey 4 
Unsat for Minor Problems--No 5 ong ( 
Impact on Product Quality 
(Time Frame--Two Years) 
Latest Quality System Review 4 
Unsat For Minor Problems 5 Or..6 
on Product Quality 
(Time Frame--Two Years) 
Latest Quality Audit/Special 4 
survey 5G <6 
Unsat for Minor Problems--No 
IMpact -On Produc: . Oude, 
(Time Frame--Two Years) 
Material Inspection Reports 4 
Reject Rate Greater Than 5% 5 <0 16 
but less than 15% Indicated 7 
(Based on Minimum of Five Lots 
in Two Years) 
First Article Test 2A 
One Unsat 4 
(Time Frame--One Year) Sor a6 

7 


ie 





ClAsotrrirCATION CRITERIA 


Quality Deficiency Reports--Cat 
One Cac. 
(Time Frame--One Year) 


Quality Deficiency Reports--Cat 
More Than Two but Less Than Six 
Cacerl 

(Time Frame--One Year) 


Report of Discrepancy 
More than Two but Less Than Six 
Contractor Liable RODs 
(Time Frame--One Year) 


Waivers/Deviations 

More Than Two but Less Than Six 
Contractor Liable Requests 
(Time Frame--One Year) 


DLA Contractor Alert List 
Listed for Minor Deficiencies-- 
No Impact on Product Quality 
(Time Frame--One Year) 


das 


ADDITIONAL 
QA REO'TS 


2A 


Sr Or. 46 


ADDITIONAL QUALITY ASSURANCE REQUIREMENTS 


Code _ and Description 


ie 


Pre-Award Survey 


A. DCAS--Quality Survey 
Be With procurement representative 
participation 


(1) Local? 
(2) Intermediate’ 
(2)" Distane 


Post-Award Orientation Conference 


A. DCAS--Quality 
B. With procurement representative 
participation 


(i) ihocaln 
(2) Intermediate” 
(3) oe Distance 


Product Oriented Survey--Procurement 
Representative with DCAS participation 
(1) loca 
(2) Intermediate’ 
(3) Dis tea 


Government Source Inspection” 


Receipt Inspection at Source--Navy 
representative with DCAS participation 
Coyiecalis 
(2) Intermediate” 
(3) Distance. 


Receipt Inspection at Destination--Navy 
representative 


(1) Low” 
(2) Medium 
(3) High’® 


Quality Assurance Letter of Instruction” 


Purchase Referrals’ 


114 


775 
1,380 
2,095 


aS), 0) 


1,075 
2,110 
3,590 


800 
1,309 
2) 2085 


300 


650 
1,360 
2, 162 


597 
1,194 
2,332 

755 


640 


LEA CALCULATIONS 


sealeculated S30/hr x 8 hrs = $240 + $35 mileage = $275 + 
DOO. 


fealeulated S30/hr x 8 hrs = $240 + $240 (8 hrs travel @ 
$30/hr) + $200 (2 days per diem @ $100/day) + $200 travel 
eects = $880 + $500. 


feaculated $30/hr x 8 hrs = $240 + $480 (16 hrs travel @ 
$30/hr) + $300 ( 3 days per diem @ $100/day) = $575 travel 
Beets = $1,595 + $500. 


“Calculated $30/hr x 16 hrs = $480 + $45 mileage = $525 + 
S550. 


feabculated $30/hr x 16 hrs = $480 + $480 (16 hrs travel @ 
$30/hr) + $400 (4 days per diem @ $100/day) + $300 travel 
Seecs = $1,660 + $550. 


"Calculated $30/hr x 16 hrs = $480 + $960 (32 hrs travel @ 
$30/hr) + $600 (6 days per diem @ $100/day) + $1,000 travel 
ees — $3,040 + $550. 


7Calculated $30/hr x 12 hrs 
$400 (DCAS costs). 


$360 + $40 mileage = $400 + 


"Calculated $30/hr x 12 hrs = $360 + $240 (8 hrs travel @ 
$30/hr) + $300 ( 3 days per diem @ $100/day) + $200 travel 
Costs = $1,100 + $400 (DCAS costs). 


“Calculated $30/hr x 12 hrs = $360 + $480 (16 hrs travel @ 
s30/hr) + $400 (4 days per diem @ $100/day) + $575 travel 
@eoes = $1,815 + $400 (DCAS costs). 


m@aqlculated $34.18/hr x 14 hrs. 


Pealculated 2Ao7 higeeeouliseses) S344 + S38) mileage = $375 + 
pe7>5 (DCAS costs). 


“calculated $43/hr x 8 hrs = $344 + $344 (8 hrs travel @ 
$43/hr) + $200 ( 2 days per diem @ $100/day) + $200 travel 
Sestes = $1,088 + $275 (DCAS costs). 


“Calculated $43/hr x 8 hrs = $344 + $688 (16 hrs travel @ 
$43/hr) + $300 ( 3 days per diem @ $100/day) + $575 travel 
Peers = $1,907 + @275 (DCAS costs). 


e@alculated $43/hr x 4 hrs = $172 + $100 material handling 
$325 test. 


Calculated $43/hr x 8 hrs 
$650 test. 


$344 + $200 material handling 


1S 


“calculated $43/hr x 24 hrs =] sieeseuer 
handling + $800 test. 


“Calculated DCAS™@)s34) le hue celnss— 
(procurement representative @ $30/hr x 


“calculated procurement representative 


1G 


$500 material 


$275 + $480 
16 hrs). 
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APPENDIX E 


Siavomomuowen ine PRICE] —~GREATEST VALUE PROCEDURES 


hemee: TO PROSPECTIVE OFFERORS (NOV 1988) 


(a) This procurement is subject to a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
program. The test is authorized by the Assistant Secretary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCS) by participating test activities. 


(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractors' past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. 


(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTORS--TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 


(d) The commodities included in this test, as currently 
solicited, are: 


poe No. FSC Nomenclature CLIN 


HM 


ADDITIONAL EVALUATION FACTORS--TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (FIXED PRICE-GREATEST VALUE PROCEDURES) 


(a) This procurement is part of a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics), for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program's effectiveness and impact on 
the acquisition process. 


(b) The purpose of R/Y/G is to assist contracting personnel 
during source selection to determine the best value for the 
Government--price, past quality performance, and other factors 
considered. The test program uses accumulated contractor 
quality performance data to classify contractor's performance 
on select commodities as either "Red" (high risk), "Yellow" 
(moderate risk), "Green" (low risk), or "Insufficient Data," 
based on the degree of risk to the Government of receiving 
poor quality products. A commodity's classification may 
change over time as new or revised quality performance data 
become available. 


(c) For purposes of source evaluation and selection, both the 
color classification of an offeror's commodity and the 
proposed price(s) shall be evaluated in accordance with 
weighted evaluation criteria established by the Government 


PY1ICr £O ThE Srecei pr om prepesalus- Price-related factors, 
such as transportation charges, discount terms, the cost of 
First Article Test, etc., will also be considered; however, 


no score or rating shall be applied. 


(d) Offerors are advised that, although price is of 
Significance in determining the successful offeror, past 
quality performance on the proposed commodity (as classified 
with the R-Y-G database) is essentially more important, and 
Shall be evaluated accordingly. Each of the R-Yyee 
classifications and its relative order of importance is 
summarized as follows: 


"Green"--Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 
Commodities within this classification are apportioned a 
greater weight or value in the evaluation than those 
classified as either "Yellow" or "Red." 


"VYellow"--Moderate risk. History of quality problems; special 
quality requirements/actions may be needed. Due to the 
additional quality assurance considerations that may be 
necessary, commodities within this classification are weighted 


ks 


less than those classified as "Green", but are of greater 
value than those within the "Red" category. 


"Red"--High risk. Special alert to history of poor quality 
performance; contract award requires higher level approval. 
These commodities are apportioned the least available weight 
Or value for past quality performance relative to commodities 
within the "Green" or "Yellow" classifications. 


"Insufficient Data"--Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history 1S unavailable; additional quality actions 
may be needed and invoked; however, commodities within this 
classification shall be evaluated solely on the basis of price 
and related factors. Past quality performance will not be a 
consideration in the evaluation of commodities for which 
current quality performance data is not set forth within the 
database. 


(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Navy Material Quality 
Assessment Office (NMQAO), Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
mo t—-9460. 


Tas 


APPENDIX F 


SAMPLE EVALUATION UNDER FIXED PRICE--GREATEST 
VALUE PROCEDURES 


Source Selection/Evaluation Method 


Total Points (Maximum) = 100 points (%) 
Total Technical =60 points (%) 

Price =40 points (%) 
Classification Adjective Numerical Rating 
Green Excellent 55-60 points 

? Good 40-54 points 
Yellow Good Boe) jevoliqnes: 
" Average 25-34 points 

‘. Marginal 16-24 points 
Red Marginal 10-15 points 
i Unacceptable O-14 points 


Evaluation Measurements 


Yellow Classification Adjective Numerical Rating 
Reject Rate = 6%-8% Good 35-39 points 
(Code G) 
Reject Rate = 9%-11% Average 25-345 peimes 
(Code G) 
Reject Rate = 12%-14% Marginal 16-24 points 
(Code G) 
3) Caveqory ss. ET 
action QDRs Good 35-39 eeinges 
(Code K) 
4 Category "II" 
action QDRs Average 25-34 points 
(Code K) 
5 Category FF" 
action QDRs Marginal 16-24 points 
(Code K) 
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